JP2013242515A - Local rhythm context imparting device, local rhythm context imparting method and program - Google Patents

Abstract

A prosodic estimation accuracy is improved and a low-cost and stable local prosodic context is provided. A local prosodic context assigning device assigns a local prosodic context for each prosodic unit to an estimation speech. . First, the prosody learning unit 110 generates a prosody model using the learning speech prosody given by analyzing the learning speech and the learning conventional context given by analyzing the learning speech. Next, the prosody estimation unit 120 generates an estimated prosody using the prosody model from the estimation conventional context generated by analyzing the estimation speech. Further, the error calculator 130 calculates an estimation error that is an error between the estimation speech prosody generated by analyzing the estimation speech and the estimation prosody. Then, the local prosodic context determination unit 140 assigns a local prosodic context when the estimation error exceeds a predetermined threshold.
[Selection] Figure 1

Description

  The present invention relates to a prosody estimation technique used for speech synthesis, and more particularly to a technique for automatically assigning a local prosodic context for estimating a prosody.

  Conventionally, in the field of speech synthesis technology, prosody estimation technology for estimating the prosody of read-out speech has been used. By combining prosody estimation technology with morphological analysis technology and accent / reading technology, which are language processing technologies, natural prosody can be estimated from the input text.

  Conventional typical prosody estimation techniques include Hidden Markov Models (Multi-Space Probability Distribution HMM, MSD-HMM) based on probabilistic distributions in multiple spaces, such as fundamental frequency (F0) and phoneme duration length. ) Based on context information such as morpheme / phoneme / accent (Non-patent Document 1) and a prosody estimation technique based on a decision tree (Non-patent Document 2).

  By the way, in the actual voice, there are various styles of voices such as a conversation style and a reading style. In order to express such various styles, prosodic contexts by various labeling methods such as emphasis and end-of-phrase tone have been devised (Non-Patent Document 3).

  In this specification, conventionally used context information such as morpheme / phoneme / accent is called “conventional context”, and context information such as emphasis / end of note tone is called “local prosodic context”.

Tokuda, Mashiko, Miyazaki, Kobayashi, “HMM based on multi-space probability distribution”, IEICE Transactions, vol.J83-D-II, No.7, pp.1579-1589, 2000. N. Iwahashi and Y. Sagisaka, “Statistical modeling of speech segment duration by constrained tree regression”, IEICE Trans., Vol.E83-D, no.7, pp.1550-1559, 2000. Maekawa, Igarashi, Kikuchi, Yoneyama, Kominato, “Intonation Labeling of the Japanese Spoken Language Corpus”, [online], [March 30, 2012 search], Internet <URL: http://www.ninjal.ac .jp / csj / manu-f / intonation.pdf>

  As described above, there are various styles of actual audio. These styles of speech are generally uttered in a certain tone, but local prosodic variations may occur depending on the meaning of the word, the function of the phrase, and the intention and focus of the speaker. . Here, when speech learning and prosody estimation are performed using only a conventional context, the information is not sufficient for modeling various styles of prosody. As a result, the accuracy of estimation is reduced as compared to the accuracy of prosody estimation in the read-out speech. Moreover, when synthesizing speech by estimating prosody from a model trained using a conventional context, the overall speech style of the learning source is reproduced, but the meaning of the word, phrase function, speaker's Since local prosodic variations due to intention, focus, etc. are not reflected, a partially uncomfortable voice is synthesized.

  In order to solve this, it is considered to give the context information described in Non-Patent Document 3 as a local prosodic context. Compared with the case where learning is performed only in the conventional context, appropriate modeling can be performed for prosody of various styles, and improvement of prosodic estimation accuracy can be expected. However, in the conventional prosody estimation technique, it is necessary to manually assign a local prosody context. Its time and cost costs are very high, and it is difficult to manually assign local prosodic contexts to all the various styles that exist in reality.

  Furthermore, even if the local prosodic context is given manually, the provision of the local prosodic context is a work based on the subjectivity of the operator, and it is difficult to unify the grant criteria. Therefore, there are many cases where the results obtained by the workers to whom the local prosodic context is given differ, and there is a problem that stable data cannot be obtained.

  The present invention has been made in view of these points, and provides a local prosodic context assignment technique that improves prosodic estimation accuracy in various styles of speech and can provide a stable local prosodic context at low cost. The purpose is to provide.

  In order to solve the above problems, the local prosodic context assigning device of the present invention assigns a local prosodic context for each prosodic unit to the estimation speech. The local prosody context assigning device includes a prosody learning unit, a prosody estimation unit, an error calculation unit, and a local prosody context determination unit. The prosody learning unit generates a prosody model using the learning speech prosody given by analyzing the learning speech and the learning conventional context given by analyzing the learning speech. The prosody estimator generates an estimated prosody using a prosodic model from the estimation conventional context given by analyzing the estimation speech. The error calculation unit calculates an estimation error which is an error between the estimation speech prosody and the estimation prosody given by analyzing the estimation speech. The local prosodic context determination unit assigns a local prosodic context when the estimation error exceeds a predetermined threshold.

  According to the local prosodic context assigning technique of the present invention, it is possible to improve the prosodic estimation accuracy by assigning a local prosodic context that can express local prosodic variations in various styles of speech. Further, since the local prosodic context can be automatically assigned, the local prosodic context can be given at a low cost. Furthermore, by providing a local prosodic context mechanically, a stable result that does not depend on the subjectivity of the operator can be obtained.

The block diagram which shows the structural example of the local prosodic context provision apparatus which concerns on 1st Embodiment. The flowchart which shows the operation example of the local prosody context provision apparatus which concerns on 1st Embodiment. The block diagram which shows the structural example of the local prosodic context provision apparatus which concerns on 2nd Embodiment. The flowchart which shows the operation example of the local prosody context provision apparatus which concerns on 2nd Embodiment. The block diagram which shows the structural example of the local prosodic context provision apparatus which concerns on 3rd Embodiment. The flowchart which shows the operation example of the local prosody context provision apparatus which concerns on 3rd Embodiment. The block diagram which shows the structural example of the local prosody context provision apparatus which concerns on 4th Embodiment. FIG. 1 is a flowchart showing an operation example of a local prosodic context assigning apparatus according to a fourth embodiment. FIG. 2 is a flowchart showing an operation example of the local prosodic context assignment apparatus according to the fourth embodiment. The block diagram which shows the structural example of the local prosodic context provision apparatus which concerns on 5th Embodiment. The flowchart which shows the operation example of the local prosody context provision apparatus which concerns on 5th Embodiment. The block diagram which shows the structural example of the local prosodic context provision apparatus which concerns on 6th Embodiment. The flowchart which shows the operation example of the local prosody context provision apparatus which concerns on 6th Embodiment. The block diagram which shows the structural example of the local prosodic context provision apparatus which concerns on 7th Embodiment. The flowchart which shows the operation example of the local prosody context provision apparatus which concerns on 7th Embodiment. The block diagram which shows the structural example of the local prosody context provision apparatus which concerns on 8th Embodiment. FIG. 1 is a flowchart showing an operation example of a local prosodic context assignment apparatus according to an eighth embodiment. FIG. 2 is a flowchart showing an operation example of the local prosodic context assignment apparatus according to the eighth embodiment.

Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.
[First Embodiment]
With reference to FIG. 1, a configuration example of a local prosodic context assignment apparatus 10 according to the first exemplary embodiment of the present invention will be described in detail. The local prosodic context assignment device 10 includes a prosody learning unit 110, a prosody estimation unit 120, an error calculation unit 130, a local prosody context determination unit 140, a learning data set storage unit 910, a prosody model storage unit 920, and an estimation data set storage unit. 930. The learning data set storage unit 910, the prosodic model storage unit 920, and the estimation data set storage unit 930 include, for example, a main storage device such as a RAM (Random Access Memory), a semiconductor memory element such as a flash memory, Alternatively, it can be configured by an auxiliary storage device such as a hard disk or an optical disk.

  With reference to FIG. 2, the operation example of the local prosodic context assignment device 10 will be described in detail according to the order of procedures actually performed.

  The learning data set storage unit 910 stores a learning data set including a learning speech prosody and a conventional learning context in advance. The speech prosody for learning is specifically the fundamental frequency (F0) or phoneme duration length of the speech for learning, and it may be automatically extracted by analyzing the speech for learning, or given manually. Also good. Specifically, the conventional learning context is a conventional context such as a morpheme, phoneme, or accent assigned to the learning speech, and the learning speech may be automatically analyzed and extracted. You may give by work. In the case of automatic extraction, a general morphological analysis technique or a reading accent analysis technique can be used. For example, the method described in Japanese Patent No. 3379643 can be applied to the morphological analysis technique. As the reading accent analysis technique, for example, a method described in Japanese Patent No. 395964 and Japanese Patent No. 3518340 can be applied. The learning voice is voice data recorded in advance, and includes voices of styles such as a conversation style and a reading style.

  The estimation data set storage unit 930 stores in advance an estimation data set including the estimation speech prosody and the estimation conventional context. The speech prosody for estimation is specifically the fundamental frequency (F0) or phoneme duration of the speech for estimation, and may be automatically extracted by analyzing the speech for estimation or given manually. Also good. Specifically, the estimation conventional context is a conventional context such as morpheme, phoneme, and accent corresponding to the estimation speech, and the estimation speech may be analyzed and automatically extracted or manually added. May be. The estimation voice is voice data recorded in advance. The estimation voice can be the same as the learning voice.

  The prosody learning unit 110 generates a prosody model by learning using the learning speech prosody stored in the learning data set storage unit 910 and the conventional learning context (S110). The prosody model learning can be performed by the method described in Non-Patent Document 1, for example. The prosody model generated by the prosody learning unit 110 is stored in the prosody model storage unit 920.

  The prosody estimation unit 120 generates an estimated prosody by estimating the prosody using the prosody model stored in the prosody model storage unit 920 from the estimation conventional context stored in the estimation data set storage unit 930. (S120). The estimated prosody is specifically the fundamental frequency (F0) of the estimation speech or the phoneme duration. If the estimation speech prosody is the fundamental frequency (F0), the estimation prosody must also be the fundamental frequency (F0). If the estimation speech prosody is a phoneme duration, the estimation prosody must also be a phoneme duration.

  The error calculation unit 130 calculates, for each prosodic unit, an estimation error that is an error between the estimation speech prosody stored in the estimation data set storage unit 930 and the estimated prosody generated by the prosody estimation unit 120 (S130). ). A prosodic unit is one of language units such as phonemes, mora, morphemes, and accent phrases. Which prosodic unit is used depends on whether the estimated speech prosody or the estimated prosody is the fundamental frequency (F0) or the phoneme duration, or what speech style is included in the training speech. , Predetermined. In general, a phoneme is often used as a prosodic unit if it is a phoneme duration, and a phoneme, mora, or accent phrase is used as a prosodic unit if it is a fundamental frequency (F0). The estimation error is one of an average value, a sum, and a maximum value of the difference between the estimation speech prosody and the estimation prosody in each prosodic unit.

The process of the error calculation unit 130 will be described more specifically by taking as an example a case where a local prosodic context is assigned to the prosodic information represented by the fundamental frequency (F0) and the prosodic unit is an accent phrase unit. The starting frame number of the i-th accent phrase _is j _is , the ending frame number is j _ie , the fundamental frequency of the estimation speech in frame j is f _o (j), and the estimated fundamental frequency (F0) is f _s (j). To do. It is assumed that the estimation speech is analyzed in units of frames in the same manner as a general fundamental frequency (F0) or spectrum analysis method. A frame window width of 30 ms and a frame shift interval of 5 ms are often used, but this is not a limitation. Here, the average error E _i in the i-th accent phrase is obtained by the following equation (1).

The error sum S _i in the i-th accent phrase is obtained by the following equation (2).

  The local prosodic context determination unit 140 compares the estimation error, which is one of the average value, the sum, and the maximum value of the difference calculated by the error calculation unit 130 for each prosodic unit, with a predetermined threshold value, and determines the estimation error. If it is equal to or greater than the threshold, a local prosodic context is assigned to the prosodic unit (S140). The local prosodic context is specifically emphasis / end-of-phrase tone, and for example, the labeling method described in Non-Patent Document 3 can be applied.

The processing of the local prosodic context determination unit 140 will be described in more detail using an example in which a local prosodic context is assigned to the prosodic information represented by the fundamental frequency (F0) and the prosodic unit is an accent phrase unit. . Let E _{i be} the average error of the i-th accent phrase and α be the threshold value. When E _i ≧ α, a local prosodic context is assigned to the i-th accent phrase. If E _i <α, no local prosodic context is assigned to the i-th accent phrase.

  As described above, the local prosodic context assigning apparatus 10 according to the first embodiment has predetermined prosodic errors generated by statistically estimating the prosody extracted from the estimation speech uttered in various styles. A local prosodic context is assigned to a prosodic unit that exceeds a threshold. Accordingly, it is possible to realize the provision of a local prosodic context that can express local prosodic variation, and to improve the prosodic estimation accuracy in various styles of speech. Further, since the local prosodic context can be automatically assigned, the local prosodic context can be given at a low cost. Furthermore, by providing a local prosodic context mechanically, a stable result that does not depend on the subjectivity of the operator can be obtained.

[Second Embodiment]
With reference to FIG. 3, a configuration example of the local prosodic context assignment apparatus 20 according to the second exemplary embodiment of the present invention will be described in detail. Similar to the local prosodic context assignment device 10 according to the first embodiment, the local prosodic context assignment device 20 includes a prosody learning unit 110, a prosody estimation unit 120, a local prosody context determination unit 140, and a learning data set storage unit 910. A prosody model storage unit 920; an error calculation unit 131 instead of the error calculation unit 130; an estimation data set storage unit 931 instead of the estimation data set storage unit 930; and a temporary local prosody context generation unit 150, a prosody re-learning unit 210, a prosody re-estimation unit 220, an error re-calculation unit 230, an optimal local prosody context determination unit 240, a prosody re-learning model storage unit 940, and a temporary local prosody context storage unit 950. The estimation data set storage unit 931, the prosody relearning model storage unit 940, and the temporary local prosody context storage unit 950 are similar to the learning data set storage unit 910 and the prosodic model storage unit 920, for example, RAM (Random Access Memory). Or a semiconductor memory element such as a flash memory, or an auxiliary storage device such as a hard disk or an optical disk.

  With reference to FIG. 4, the operation example of the local prosodic context assignment device 20 will be described in detail according to the order of procedures actually performed. Since the processing from S110 to S120 is the same as that of the first embodiment, description thereof is omitted here.

  In addition to the information stored in the estimation data set storage unit 930, the estimation data set storage unit 931 stores an estimation local prosodic context. Specifically, the local prosodic context for estimation is a local prosodic context such as emphasis and end-of-speech tone that is manually or mechanically assigned to the estimation speech.

The error calculation unit 131 is different from the error calculation unit 130 in that the calculation formulas for the average error and the error sum are different. Similar to the first embodiment, the processing of the error calculation unit 131 is exemplified by a case where a local prosodic context is assigned to the prosodic information represented by the fundamental frequency (F0) and the prosodic unit is an accent phrase unit. Will be described in detail. The starting frame number of the i-th accent phrase _is j _is , the ending frame number is j _ie , the fundamental frequency of the estimation speech in frame j is f _o (j), and the estimated fundamental frequency (F0) is f _s (j). Then, the average error E ′ _i in the i-th accent phrase is obtained by the following equation (1 ′).

The error sum S ′ _i in the i-th accent phrase is obtained by the following equation (2 ′).

This is because the sign of the difference has an important meaning at the stage of determining the threshold.

The temporary local prosodic context generation unit 150 executes the processing of the local prosodic context determination unit 140 for each of a plurality of predetermined thresholds, and includes a temporary local prosodic context set including a temporary local prosodic context that is a local prosodic context corresponding to each threshold. Is generated. First, the temporary local prosodic context generation unit 150 sets the threshold referred to by the local prosodic context determination unit 140 to a threshold that is greater than or equal to a predetermined start value and within an end value (S151). More specifically, when the threshold start value is α _s , the end value is α _e , and the increment is Δα, the threshold value α set in the n-th iteration can be expressed by the following equation (3).

Here, the values of the start value α _s , the end value α _e , and the increment Δα differ depending on whether the prosody is the fundamental frequency (F0) or the phoneme duration. For example, if the prosody is the fundamental frequency (F0), the start value α _s can be 0 cent, the end value α _e can be 200 cent, and the increment Δα can be 1 cent.

Subsequently, the temporary local prosodic context generation unit 150 executes the processing of the local prosodic context determination unit 140, and sets the local prosodic context generated by the local prosodic context determination unit 140 as a temporary local prosodic context (S140). Thereafter, the temporary local prosodic context storage unit 950 stores the temporary local prosodic context (S152). Temporary local prosodic context generator 150, the threshold value alpha is S151 until the end value alpha _e, S140, S152 repeatedly executed (S155). As a result, a temporary local prosodic context set including all temporary local prosodic contexts obtained at all threshold values not less than the start value α _{s and} within the end value α _e is stored in the temporary local prosodic context storage unit 950. For example, if the R-th threshold is set, the temporary local prosodic context set includes R temporary local prosodic contexts.

  The prosody re-learning unit 210 selects a temporary local prosodic context selected from the temporary local prosody context set stored in the temporary local prosody context storage unit 950, and the estimation speech prosody and estimation stored in the estimation data set storage unit 930. The prosody re-learning model is generated by learning using the conventional context (S210). Learning of the prosody relearning model can be performed by a method similar to that of the prosody learning unit 110. The prosody relearning model generated by the prosody relearning unit 210 is stored in the prosody relearning model storage unit 940.

  The prosody re-estimation unit 220 uses the estimation local prosodic context and prosody re-learning model storage unit 940 stored in the estimation data set storage unit 931 from the estimation conventional context stored in the estimation data set storage unit 931. A re-estimation prosody is generated by estimating the prosody using the prosody re-learning model stored in (S220). Here, as the local prosodic context for estimation, a temporary local prosodic context arbitrarily selected from the temporary local prosodic context set can be used instead of the local prosodic context given manually or mechanically. In this case, the local prosodic context assignment device 20 may include the estimation data set storage unit 930 included in the local prosody context assignment device 10 according to the first embodiment instead of the estimation data set storage unit 931.

The error recalculation unit 230 calculates a re-estimation error that is an error between the estimation speech prosody stored in the estimation data set storage unit 931 and the re-estimation prosody generated by the prosody re-estimation unit 220 for each prosodic unit. Calculate (S230). The re-estimation error is any one of an average value E _a , a total sum E _s, and a maximum value of the difference between the estimation speech prosody and the re-estimation prosody in each prosody unit. If the start frame number of the estimation speech is j _s , the end frame number is j _e , the fundamental frequency of the estimation speech at frame number j is f _o (j), and the estimated fundamental frequency (F0) is f _s (j) The average value E _a of the difference between the estimation speech prosody and the re-estimation prosody is obtained by the following equation (4).

The sum E _s of the difference between the estimation speech prosody and the re-estimation prosody is obtained by the following equation (5).

The optimal local prosodic context determination unit 240 determines that the re-estimation error E calculated by the error recalculation unit 230 is the smallest among the temporary local prosodic contexts included in the temporary local prosodic context set stored in the temporary local prosodic context storage unit 950. Is selected as the local prosodic context. More specifically, the optimal local prosodic context determination unit 240 stores the minimum value E _min of the re-estimation error, and the re-estimation error E calculated by the error re-calculation unit 230 is greater than the previous minimum value E _min . If the value is also smaller, the minimum value E _min is updated. That is, E _min = E when E <E _min . If there is a temporary local prosodic context for which the re-estimation error E is not found among the temporary local prosodic contexts included in the temporary local prosodic context set stored in the temporary local prosodic context storage unit 950, the temporary local prosodic context is determined. A prosodic context is selected and the processing from S210 to S230 is executed again (S235). When the re-estimation error E is obtained for all temporary local prosodic contexts, the temporary local prosodic context with the re-estimation error having the minimum value E _min is selected as the optimal local prosodic context (S240).

  In the present invention, since the error threshold value is used as a condition for determining whether or not the local prosodic context determination unit 140 assigns the local prosodic context, the local prosodic context obtained as a result varies depending on the threshold value. The local prosodic context assigning apparatus 20 according to the second embodiment generates a temporary local prosodic context corresponding to each of a plurality of predetermined threshold values, and has the smallest error compared to a preliminarily given estimation local prosodic context. By selecting the temporary local prosodic context, it is possible to obtain a local prosodic context based on an optimum threshold that best reflects the local prosodic variation in various styles of speech.

[Third Embodiment]
With reference to FIG. 5, a configuration example of the local prosodic context assignment device 30 according to the third exemplary embodiment of the present invention will be described in detail. Similar to the local prosodic context assignment device 20 according to the second embodiment, the local prosodic context assignment device 30 is a prosodic learning unit 110, a prosody estimation unit 120, an error calculation unit 131, a local prosodic context determination unit 140, and a provisional local prosody. A context generation unit 150, a prosody re-learning unit 210, an error re-calculation unit 230, a learning data set storage unit 910, a prosody model storage unit 920, an estimation data set storage unit 931, and a prosody re-learning model storage unit 940; A prosody re-estimation unit 221 is provided instead of the re-estimation unit 220, and an optimal local prosody context determination unit 241 is provided instead of the optimal local prosody context determination unit 240.

  With reference to FIG. 6, the operation example of the local prosodic context assigning device 30 will be described in detail according to the order of procedures actually performed. Since the processing from S110 to S210 is the same as that of the second embodiment, the description thereof is omitted here.

  The prosody re-estimation unit 221 estimates the prosody from the estimation conventional context stored in the estimation data set storage unit 931 using the prosody re-learning model stored in the prosody re-learning model storage unit 940. Then, a re-estimated prosody is generated (S221). That is, the estimation local prosodic context used when estimating the prosody in the second embodiment is not required.

The optimal local prosodic context determining unit 241 determines that the re-estimation error E calculated by the error recalculating unit 230 is the largest among the temporary local prosodic contexts included in the temporary local prosodic context set stored in the temporary local prosodic context storage unit 950. Is selected as the local prosodic context. More specifically, the optimal local prosodic context determination unit 241 stores the maximum value E _max of the re-estimation error E, and the re-estimation error E calculated by the error re-calculation unit 230 is the maximum value E _max so far. If the value is larger than the maximum value, the maximum value E _max is updated. That is, E _max = E when E> E _max . If there is a temporary local prosodic context for which the re-estimation error E is not found among the temporary local prosodic contexts included in the temporary local prosodic context set stored in the temporary local prosodic context storage unit 950, the temporary local prosodic context is determined. A prosodic context is selected and the processing from S210 to S230 is executed again (S235). When the re-estimation error E is obtained for all the temporary local prosodic contexts, the temporary local prosodic context with the re-estimation error E having the maximum value E _max is selected as the optimal local prosodic context (S241).

  Similar to the local prosodic context assigning apparatus 20 according to the second embodiment, the local prosodic context assigning apparatus 30 according to the third embodiment is the best reflecting the local prosody variation in various styles of speech. A local prosodic context based on a certain threshold value can be obtained. Which embodiment is selected depends on whether the target prosody is the fundamental frequency (F0) or the phoneme duration, what kind of speech style is included in the learning speech, etc. It can be selected in consideration.

[Modification]
In the third embodiment, an optimal local prosodic context determining unit 242 may be provided instead of the optimal local prosodic context determining unit 241. The optimal local prosodic context determination unit 241 determines a temporary prosodic context that maximizes the average value, sum, or maximum value of the difference between the estimated speech prosody and the re-estimated prosody in the entire estimation speech. In the optimal local prosodic context determining unit 242, the sum of any one of the average value, the sum, and the maximum value of the difference between the speech prosody for estimation and the re-estimated prosody in the prosodic unit to which the temporary local prosodic context is assigned is selected. Alternatively, the absolute value of the difference between the average value and the sum of the average value, sum, or maximum value of the difference between the prosody for estimation and the re-estimation prosody in the prosodic unit to which the temporary local prosodic context is not assigned The temporary local prosodic context with the maximum error E is selected as the optimal local prosodic context with the value as the error E.

A method for calculating the error E will be specifically described. Assume that m prosodic units are included in the speech prosody for estimation, and provisional local prosodic contexts are assigned to k prosodic units. A set of numbers of prosodic units to which the temporary local prosodic context is assigned is defined as PU _e . That is, | PU _e | = k. A set of prosodic unit numbers to which no provisional local prosodic context is assigned is represented by PU _p . That is, | PU _p | = mk. Here, ・ ￣ indicates that the preceding character is overlined, and the average value E￣ _em of the provisional local prosodic context is given by the following equation (6): Desired. Here, E ′ _i is calculated based on the definition of equation (1 ′).

The average value E￣ _p of the prosodic units to which the temporary local prosodic context is not given is obtained by the following equation (7).

Then, the error E, using the average value E _p and the average value E _em, is determined by the following equation (8).

[Fourth Embodiment]
With reference to FIG. 7, a configuration example of the local prosodic context assignment apparatus 40 according to the fourth exemplary embodiment of the present invention will be described in detail. Similar to the local prosodic context assigning apparatus 20 according to the second embodiment, the local prosodic context assigning apparatus 40 includes a prosody learning unit 110, a prosody estimation unit 120, an error calculation unit 131, a local prosodic context determination unit 140, and a provisional local prosody. Context generation unit 150, prosody re-learning unit 210, prosody re-estimation unit 220, error recalculation unit 230, optimal local prosody context determination unit 240, learning data set storage unit 910, prosodic model storage unit 920, and estimation data set storage 931, a prosodic relearning model storage unit 940, and a temporary local prosody context storage unit 950, and further, a local prosodic context candidate setting unit 310, a local prosodic context candidate update unit 320, a local prosodic context candidate determination unit 330, and a temporary local prosody A context update unit 340 and a re-estimation error storage unit 960 are provided. . The re-estimation error storage unit 960 is similar to the learning data set storage unit 910, prosody model storage unit 920, estimation data set storage unit 931, prosody re-learning model storage unit 940, and temporary local prosody context storage unit 950, for example. It can be constituted by a main storage device such as a RAM (Random Access Memory), a semiconductor memory device such as a flash memory, or an auxiliary storage device such as a hard disk or an optical disk.

  With reference to FIGS. 8 and 9, an operation example of the local prosodic context assigning device 40 will be described in detail in the order of procedures actually performed. This represents that the flow of processing continues from A in FIG. 8 to A in FIG. Since the processing from S110 to S155 is the same as that of the second embodiment, the description thereof is omitted here.

  The local prosodic context candidate setting unit 310 arbitrarily selects a temporary local prosodic context from the temporary local prosodic context set stored in the temporary local prosodic context storage unit 950 and sets it as a local prosodic context candidate. Subsequently, the selected temporary local prosodic context is deleted from the temporary local prosodic context set (S310).

  The local prosodic context candidate update unit 320 compares the previous re-estimation error stored in the re-estimation error storage unit 960 with the re-estimation error calculated by the error re-calculation unit 230 for each prosodic unit. When the re-estimation error is larger than the previous re-estimation error, and when the local prosodic context corresponding to the prosodic unit is assigned to the local prosodic context candidate set by the local prosodic context candidate setting unit 310, the local prosody Delete the context. If the local prosodic context is not assigned to the local prosodic context candidate, the local prosodic context is assigned. If the re-estimation error is less than or equal to the previous re-estimation error, the local prosodic context is not changed (S320). For example, for the i-th accent phrase, assuming that a local prosodic context is given in the (n-1) th iteration, the error in the nth iteration of the i-th accent phrase is e (n), the threshold is α, the i-th accent phrase Assuming that the error in the n-1th iteration of the accent phrase is e (n-1), the local prosodic context of the accent phrase is not changed when e (n) -e (n-1) ≤0. That is, the local prosodic context remains attached. If e (n) -e (n-1)> 0, the local prosodic context of the accent phrase is deleted. The local prosodic context candidate update unit 320 determines the presence / absence of a local prosodic context for all prosodic units included in the speech prosody for estimation. After determining whether or not there is a local prosodic context for all prosodic units, the re-estimation error stored in the re-estimation error storage unit 960 is updated and stored with the re-estimation error calculated by the error recalculation unit 230.

  The local prosodic context candidate determination unit 330 executes the prosody re-learning unit 210, the prosody re-estimation unit 220, the error recalculation unit 230, and the local prosody context candidate update unit 320 (S330). Here, if the number of repetitions is equal to or less than a predetermined number and the re-estimation error in the entire estimation speech prosody is equal to or greater than a predetermined value, the repetition prosody re-learning unit 210, the prosody re-estimation unit 220, The calculation unit 230 and the local prosodic context candidate update unit 320 are executed. When the number of repetitions exceeds a predetermined number, or when the re-estimation error in the entire estimation speech prosody is smaller than a predetermined value, the processing of the local prosodic context candidate determination unit 330 is terminated (S325). ).

  The temporary local prosodic context update unit 340 repeatedly executes the processes of the local prosodic context candidate setting unit 310 and the local prosodic context candidate determination unit 330 (S340). Here, when the local prosodic context candidate setting unit 310 stops setting local prosodic context candidates, that is, for all temporary local prosodic contexts included in the temporary local prosodic context set stored in the temporary local prosodic context storage unit 950. When the local prosodic context candidate is determined, the process of the temporary local prosodic context update unit 340 is terminated (S335).

  The optimal local prosodic context determination unit 240 selects the local prosodic context candidate that minimizes the re-estimation error calculated by the error recalculation unit 230 among the local prosodic context candidates corresponding to all temporary local prosodic contexts. (S240).

  The local prosodic context assigning apparatus 40 according to the fourth embodiment repeatedly performs re-learning and re-estimation for all provisional local prosodic contexts until the estimation speech prosody as a whole is equal to or less than a predetermined threshold value. Is selected as the optimal local prosodic context. By configuring in this way, it is possible to determine the optimum threshold value more strictly than in the second embodiment.

[Fifth Embodiment]
With reference to FIG. 10, a configuration example of the local prosodic context assignment apparatus 50 according to the fifth exemplary embodiment of the present invention will be described in detail. Similar to the local prosodic context assignment device 10 according to the first embodiment, the local prosodic context assignment device 50 estimates the prosody learning unit 110, the prosody estimation unit 120, the learning data set storage unit 910, and the prosody model storage unit 920. A data set storage unit 930, an error calculation unit 131 according to the second embodiment, and a local prosody context determination unit 141 instead of the local prosody context determination unit 140.

  With reference to FIG. 11, an operation example of the local prosodic context assignment device 50 will be described in detail according to the order of procedures actually performed. Since the processing from S110 to S131 is the same as that of the second embodiment, description thereof is omitted here.

  The local prosodic context determination unit 141 compares the estimation error, which is one of the average value, the sum, and the maximum value of the difference calculated by the error calculation unit 131 for each prosodic unit with a plurality of predetermined threshold values. A range of threshold values to which the estimation error applies is obtained, and a local prosodic context class corresponding to the threshold is assigned to the prosodic unit (S141).

More specifically, the local prosodic context determination unit 141 performs processing by giving a local prosodic context class value to the prosodic information represented by the fundamental frequency (F0) and using the prosodic unit as an accent phrase unit. Explained. For example, if the local prosodic context is divided into n classes (n is an integer of 3 or more), the class set CL = {C ₁ , C ₂ ,..., C _n } of _n local prosodic contexts and n− One threshold value α ₁ , α ₂ ,..., Α _n-1 is given. However, α ₁ <α ₂ <α ₃ ... <α _n−1 . The threshold _values α ₁ , α ₂ ,..., Α _n−1 correspond to the class set CL = {C ₁ , C ₂ ,..., C _n }, respectively, and the class C _i (i = 1,. The threshold corresponding to 1) is α _i . At this time, the threshold value corresponding to the class C _n is ∞.

Here, assuming that the estimation error of the j-th accent phrase is E _j , the estimation error E _j is compared with the thresholds α ₁ , α ₂ ,..., Α _n−1, and α _k-1 ≦ E _j <α _k Find the value of k such that (k is an integer between 2 and n-1). If the value of k is obtained, the class of the local prosodic context of the jth accent phrase is C _k . In particular, when E _j <α ₁ , the class of the local prosodic context is C ₁ . When α _n-1 ≦ E _j , the local prosodic context class is C _n .

Note that when n is 2 and the local prosodic context is divided into two classes, the threshold value is only α _1, and thus the configuration is the same as that of the first embodiment.

  As described above, the local prosodic context assigning device 50 according to the fifth embodiment uses the prosody error generated by statistically estimating the prosody extracted from the estimation speech uttered in one or a plurality of styles. Based on the predetermined threshold range for each class, a local prosodic context class is assigned to each prosodic unit. Thus, unlike the binary context assignment of whether or not the local prosodic context is given in the first embodiment, the local prosody expressed in detail by dividing the variation of the local prosody into a plurality of stages. A context can be granted. Therefore, it is possible to further improve the accuracy of prosody estimation in a plurality of styles of speech.

[Sixth Embodiment]
With reference to FIG. 12, a configuration example of a local prosodic context assignment device 60 according to the sixth exemplary embodiment of the present invention will be described in detail. Similar to the local prosodic context assignment device 20 according to the second embodiment, the local prosodic context assignment device 60 includes a prosody learning unit 110, a prosody estimation unit 120, an error calculation unit 131, a prosody re-learning unit 210, and a prosody re-estimation unit. 220, error recalculation unit 230, optimum local prosodic context determination unit 240, learning data set storage unit 910, prosody model storage unit 920, estimation data set storage unit 931, prosody re-learning model storage unit 940, and provisional local prosodic context A storage unit 950; a local prosodic context determination unit 141 according to the fifth embodiment; and a temporary local prosodic context generation unit 151 instead of the temporary local prosody context generation unit 150.

  With reference to FIG. 13, an operation example of the local prosodic context assignment device 60 will be described in detail according to the order of procedures actually performed. Since the processing from S110 to S141 is the same as that of the fifth embodiment, the description thereof is omitted here.

The temporary local prosodic context generation unit 151 executes the processing of the local prosodic context determination unit 141 for each of a plurality of predetermined threshold values, and sets a temporary local prosodic context set including a temporary local prosodic context that is a local prosodic context corresponding to each threshold value. Is generated. First, the temporary local prosodic context generation unit 151 sets a threshold corresponding to each class of the local prosodic context referred to by the local prosodic context determination unit 141 (S153). Assuming that the local prosodic context is divided into n classes (n is an integer of 3 or more), and the thresholds corresponding to each class are α ₁ , α ₂ ,..., Α _n−1 , α ₁ <α ₂ … <α _The threshold values α ₁ , α ₂ ,..., α _n-1 may be set to any value as long as they are in a relation of _n−1 and are in a reasonable range that can actually take an error.

A method for setting the threshold _values α ₁ , α ₂ ,..., Α _n−1 will be described in more detail. For example, the minimum value of error is Min, the maximum value is Max, α ₁ is determined to be an appropriate value between Min and Max, α ₂ is determined to be between α _{1 and} Max, and this is repeated. A method of determining all n-1 threshold values simultaneously is conceivable. With this method, it is possible to search for all possible combinations of n−1 threshold values, and it is best in terms of searching for an optimum value. However, there is a problem that it is necessary to set the number of classes appropriately, and at the same time, the amount of calculation becomes very large.

  Therefore, considering the fact that the actual error distribution is close to the normal distribution centered on 0, the threshold is set to be the same in the positive and negative directions centered on 0, and it is appropriate if the number of data corresponding to each class is small. In consideration of the points that cannot be learned, a method is conceivable in which the threshold is sequentially set based on the threshold and the number of data for each class without fixing the number of classes.

For example, β _j is a positive real number, m is a positive integer, and thresholds are −β _m , −β _m−1 , −β _m−2 ,..., −β ₁ , β ₁ , β _{2 ,.} And Here, m represents an upper limit value of the predetermined number of classes.

First, the number of classes is set to 3 classes. When the estimation error E is E <−β ₁ , class C ₁ is satisfied, and when −β ₁ ≦ E <β ₁ is satisfied, class C ₂ and β ₁ ≦ E are satisfied. In this case, class C ₃ . The beta ₁ starting value of the time threshold value β ^s _1, β ^e ₁ the end value, when the increment and [Delta] [beta] _1, in L ₁ th iteration, is possible to set a threshold beta ₁ by the following equation (9) it can.

Here, for example, if it is known in advance that at least Q pieces of data are required for each class in order for the learning in the prosody re-learning unit 210 to be performed appropriately, the start value β ^s a value of ₁ may be a value that accent phrase number estimation error becomes larger beta ^s ₁ smaller value than-beta ^s ₁ is the Q. The value of the increment Δβ ₁ varies depending on whether the prosody is the fundamental frequency (F0) or the phoneme duration length.For example, if the prosody is the logarithmic fundamental frequency (F0), the increment Δβ ₁ may be ₁ cent. it can. The value of the end value β ^e _{1 is} a value that results in Q accent phrases with an estimation error smaller than −β ^e ₁ or larger than β ^e ₁ .

Next, when the number of classes is five and the estimation error E is E <−β ₂ , class C ₁ , and when −β ₂ ≦ E <−β ₁ is satisfied, class C ₂ , −β ₁ ≦ E < When β ₁ , class C ₃ , when β ₁ ≦ E <β ₂ , class C ₄ , and when β ₂ ≦ E, class C ₅ . In L ₂ iteration based on beta ₂ threshold to beta _1, it is set by the following equation (10). At this time, β ₁ is fixed at the value determined as described above.

Here, the value of the start value beta ^s ₂ is the estimated error-beta ^s ₂ greater than and-beta ^s ₁ below, or beta ^s ₁ becomes larger than and beta ^s ₂ the following values, the number of the accent phrase into Q Value. The value of the increment Δβ ₂ may be the same as or different from Δβ ₁ . The value of the end value β ^e _{2 is} a value where the estimation error is smaller than −β ^e ₁ or larger than β ^e ₁ and the number of accent phrases is Q.

Hereinafter, similarly, the value of β _j is determined while increasing the number of classes by two. If a certain β _j value is less than -β ^e _j or larger than β ^e _j and the number of accent phrases is Q or less, the number of classes is 2j-1 and β _j is not set. Shall.

  In the above example, the threshold value is sequentially determined based on the number Q necessary for the learning to be properly performed. However, when other appropriate values are known in advance, the value is used instead of Q. May be given. In particular, when appropriate values are known as the start value and end value of the threshold value for each class regardless of the number of data for each class, the values may be given in advance.

  By setting the threshold value as described above, stable learning can be performed while suppressing the amount of calculation.

  Subsequently, the temporary local prosodic context generation unit 151 executes the processing of the local prosodic context determination unit 141, and based on the local prosodic context class assigned to the prosodic unit by the local prosodic context determination unit 141, Is generated (S141). Thereafter, the temporary local prosodic context storage unit 950 stores the temporary local prosodic context (S154). The temporary local prosodic context generation unit 151 repeatedly executes S153, S141, and S154 until calculation is made for all possible threshold combinations (S156).

  Since the subsequent processing is the same as that of the second embodiment, description thereof is omitted here.

  The local prosodic context assigning device 60 according to the sixth embodiment generates temporary local prosodic contexts corresponding to respective classes of local prosodic contexts corresponding to a plurality of predetermined threshold values, and preliminarily given local prosodic contexts for estimation By selecting the temporary local prosodic context with the smallest error compared to the local prosodic context, it is possible to obtain a local prosodic context based on the optimum threshold that best reflects the local prosodic variation in various styles of speech.

[Seventh Embodiment]
With reference to FIG. 14, a configuration example of a local prosodic context assignment device 70 according to the seventh exemplary embodiment of the present invention will be described in detail. Similar to the local prosodic context assignment device 30 according to the third embodiment, the local prosodic context assignment device 70 is a prosody learning unit 110, a prosody estimation unit 120, an error calculation unit 131, a prosody re-learning unit 210, and a prosody re-estimation unit. 221, error recalculation unit 230, optimal local prosody context determination unit 241, learning data set storage unit 910, prosody model storage unit 920, estimation data set storage unit 931, and prosody re-learning model storage unit 940, The local prosody context determination unit 141 according to the embodiment and the temporary local prosody context generation unit 151 according to the sixth embodiment are provided.

  FIG. 15 shows an operation example of the local prosodic context assigning apparatus 70 according to the seventh embodiment. Since the processing of each step is the same as the description of the third embodiment, the fifth embodiment, and the sixth embodiment, the description is omitted here.

  Similar to the local prosodic context assigning apparatus 60 according to the sixth embodiment, the local prosodic context assigning apparatus 70 according to the seventh embodiment optimally reflects local prosody variations in various styles of speech. A local prosodic context based on a certain threshold value can be obtained. Which embodiment is selected depends on whether the target prosody is the fundamental frequency (F0) or the phoneme duration, what kind of speech style is included in the learning speech, etc. It can be selected in consideration.

[Eighth Embodiment]
With reference to FIG. 16, a configuration example of a local prosodic context assignment device 80 according to the eighth exemplary embodiment of the present invention will be described in detail. Similar to the local prosodic context assignment device 40 according to the fourth embodiment, the local prosodic context assignment device 80 is a prosody learning unit 110, a prosody estimation unit 120, an error calculation unit 131, a prosody re-learning unit 210, and a prosody re-estimation unit. 220, error recalculation unit 230, optimal local prosody context determination unit 240, local prosody context candidate setting unit 310, local prosody context candidate update unit 320, local prosody context candidate determination unit 330, provisional local prosody context update unit 340, and learning A fifth embodiment includes a data set storage unit 910, a prosody model storage unit 920, an estimation data set storage unit 931, a prosody re-learning model storage unit 940, a temporary local prosody context storage unit 950, and a re-estimation error storage unit 960. Local prosodic context determination unit 141 according to the present invention, and temporary local according to the sixth embodiment It comprises law context generating unit 151.

  17 and 18 show an operation example of the local prosodic context assignment device 80 according to the eighth embodiment. Since the process of each step is the same as the description of the fourth embodiment, the fifth embodiment, and the sixth embodiment, the description is omitted here.

  The local prosodic context assigning apparatus 80 according to the eighth embodiment repeatedly performs re-learning and re-estimation for all provisional local prosodic contexts until the estimation speech prosody as a whole is equal to or less than a predetermined threshold value. Is selected as the optimal local prosodic context. By configuring in this way, it is possible to determine the optimum threshold value more strictly than in the sixth embodiment.

[Program, recording medium]
The present invention is not limited to the above-described embodiment, and it goes without saying that modifications can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above-described embodiments are not only executed in time series according to the order described, but may be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes.
When various processing functions in each device described in the above embodiment are realized by a computer, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.
The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

10, 20, 30, 40, 50, 60, 70, 80 Local prosody context assigning device 110 Prosody learning unit 120 Prosody estimation unit 130 Error calculation unit 140, 141 Local prosody context determination unit 150, 151 Temporary local prosody context generation unit 210 Prosody re-learning unit 220, 221 Prosody re-estimation unit 230 Error recalculation unit 240, 241 Optimal local prosody context determination unit 310 Local prosody context candidate setting unit 320 Local prosody context candidate update unit 330 Local prosody context candidate determination unit 340 Temporary local prosody Context update unit 910 Learning data set storage unit 920 Prosody model storage unit 930, 931 Estimation data set storage unit 940 Prosody re-learning model storage unit 950 Temporary local prosody context storage unit 960 Re-estimation error storage unit

Claims (9)

A local prosodic context assignment device that assigns a local prosodic context for each prosodic unit to a speech for estimation,
A prosody learning unit that generates a prosody model using the learning speech prosody given by analyzing the learning speech and the learning conventional context given by analyzing the learning speech;
A prosodic estimation unit that generates an estimated prosody using the prosodic model from the conventional context for estimation given by analyzing the estimation speech;
An error calculation unit for calculating an estimation error that is an error between the estimated speech prosody generated by analyzing the estimated speech and the estimated prosody;
A local prosodic context determination unit that assigns a local prosodic context using the estimation error and a predetermined threshold;
A local prosodic context assigning device comprising: The local prosodic context assigning device according to claim 1,
The local prosodic context assignment device, wherein the local prosodic context determination unit assigns a local prosodic context when the estimation error exceeds the threshold. The local prosodic context assigning device according to claim 1,
A local prosodic context assignment device, wherein the local prosodic context determination unit assigns a class of a local prosodic context according to a range of threshold values in which a plurality of threshold values are set and the estimation error is included. The local prosodic context giving device according to any one of claims 1 to 3,
The local prosodic context determination unit executes the processing for each of a plurality of predetermined threshold values, and generates a temporary local prosodic context set including all temporary local prosodic contexts using the local prosodic context corresponding to each threshold as a temporary local prosodic context. A provisional local prosodic context generation unit,
For each temporary local prosodic context, a prosody relearning unit that generates a prosody relearning model using the estimation conventional context;
For each temporary local prosodic context, a prosody re-estimation unit that generates a re-estimated prosody using at least the prosody relearning model;
An error recalculation unit that calculates a re-estimation error that is an error between the estimation speech prosody and the re-estimation speech prosody in the entire estimation speech for each temporary local prosody context;
An optimal local prosodic context determining unit that selects the local prosodic context from the temporary local prosodic context based on the re-estimation error;
A local prosodic context assigning device comprising: The local prosodic context assignment device according to claim 4,
The prosody re-estimator is
For each of the temporary local prosodic contexts, the re-estimated prosody is generated using the pre-estimated local prosodic context and the prosodic relearning model,
The optimal local prosodic context determining unit includes:
The provisional local prosodic context that minimizes the re-estimation error is selected as the local prosodic context. The local prosodic context assignment device according to claim 4,
The prosody re-estimator is
For each temporary local prosodic context, generate the re-estimated prosody using the prosody relearning model;
The optimal local prosodic context determining unit includes:
The provisional local prosodic context that maximizes the re-estimation error is selected as the local prosodic context. The local prosodic context giving device according to claim 5,
A re-estimation error storage unit that stores a re-estimation error calculated by the error re-calculation unit as a previous re-estimation error;
A temporary local prosodic context arbitrarily selected from the temporary local prosodic context set is set as a local prosodic context candidate, and a local prosodic context candidate setting unit for deleting the temporary local prosodic context from the temporary local prosodic context set;
When the previous re-estimation error is compared with the re-estimation error and the re-estimation error is larger and the local prosodic context is given to the local prosodic context candidate, the local prosodic context And when the local prosodic context is not given to the local prosodic context candidate, a local prosodic context candidate update unit for giving the local prosodic context;
A local prosody context candidate determination unit that repeatedly executes the prosody re-learning unit, the prosody re-estimation unit, the error recalculation unit, and the local prosody context candidate update unit until a predetermined condition is satisfied;
A temporary local prosodic context update unit that repeatedly executes the processes of the local prosodic context candidate setting unit and the local prosodic context candidate determination unit until all temporary local prosodic contexts are deleted from the temporary local prosodic context set;
A local prosodic context assigning device comprising: A local prosodic context giving method for giving a local prosodic context for each prosodic unit to a speech for estimation,
A prosodic learning step for generating a prosodic model using the learning speech prosody given by analyzing the learning speech and the learning conventional context given by analyzing the learning speech;
A prosody estimation step for generating an estimated prosody using the prosodic model from the estimation conventional context generated by analyzing the estimation speech;
An error calculating step of calculating an estimation error that is an error between the estimated speech prosody generated by analyzing the estimated speech and the estimated prosody;
A local prosodic context determination step for assigning a local prosodic context using the estimation error and a predetermined threshold;
A local prosodic context assigning method characterized by comprising:   A program that causes a computer to function as the local prosodic context assignment device according to claim 1.

Images