WO2024051481A1 - 音频处理方法、装置、设备、可读存储介质及程序产品 - Google Patents

音频处理方法、装置、设备、可读存储介质及程序产品 Download PDF

Info

Publication number
WO2024051481A1
WO2024051481A1 PCT/CN2023/114040 CN2023114040W WO2024051481A1 WO 2024051481 A1 WO2024051481 A1 WO 2024051481A1 CN 2023114040 W CN2023114040 W CN 2023114040W WO 2024051481 A1 WO2024051481 A1 WO 2024051481A1
Authority
WO
WIPO (PCT)
Prior art keywords
similarity
similarity matrix
audio
matrix
row
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/114040
Other languages
English (en)
French (fr)
Inventor
朱鸿宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to EP23862178.3A priority Critical patent/EP4586105A4/en
Priority to US18/583,688 priority patent/US20240242722A1/en
Publication of WO2024051481A1 publication Critical patent/WO2024051481A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
    • G06F16/65—Clustering; Classification
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00—Speaker identification or verification techniques
    • G10L17/02—Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00—Speaker identification or verification techniques
    • G10L17/04—Training, enrolment or model building
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00—Speaker identification or verification techniques
    • G10L17/06—Decision making techniques; Pattern matching strategies
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination

Definitions

  • the embodiments of the present application relate to the field of computer technology, and in particular to an audio processing method, device, equipment, readable storage medium, and program product.
  • audio object clustering is an audio processing method. Audio object clustering is to determine the number of audio objects included in multiple audio segments and the audio segments corresponding to each audio object.
  • multiple audio clips are obtained, the voiceprint vectors corresponding to each audio clip are determined, the number of audio objects existing in the multiple audio clips is determined based on the similarity between the voiceprint vectors, and the number of audio objects is determined based on the number of audio objects.
  • Multiple audio clips are clustered to obtain the audio clips corresponding to each audio object.
  • Embodiments of the present application provide an audio processing method, device, equipment, readable storage medium, and program product, which can improve the accuracy of audio object clustering.
  • embodiments of the present application provide an audio processing method, which is executed by a computer device.
  • the method includes:
  • the initial similarity matrix is adjusted to obtain a reference similarity matrix, and the dynamic threshold is used to adjust the similarity difference between different similarities;
  • the plurality of audio segments are clustered to obtain audio segments resulting from the sounds of each audio object.
  • an audio processing device which includes:
  • a determination module configured to determine voiceprint vectors corresponding to multiple audio segments, where the voiceprint vectors are used to represent voiceprint features corresponding to the audio segments;
  • the determination module is also configured to determine an initial similarity matrix based on the voiceprint vectors corresponding to each audio segment, where the initial similarity matrix includes the similarity between the voiceprint vectors corresponding to any two audio segments;
  • An adjustment module configured to adjust the initial similarity matrix according to the dynamic threshold corresponding to each row in the initial similarity matrix to obtain a reference similarity matrix.
  • the dynamic threshold is used to adjust the similarity between different similarities. Adjust the difference;
  • the determination module is also configured to determine the audio of the plurality of audio segments based on the reference similarity matrix. number of objects;
  • a clustering module is configured to cluster the plurality of audio clips according to the number of the audio objects to obtain the audio clips produced by the sound produced by each audio object.
  • the determination module is further configured to, for any row in the initial similarity matrix, perform a first order on the similarities in the any row that are within a preset similarity range. Sort to obtain a first sorting result; determine the similarity difference between two adjacent similarities in the first sorting result to obtain multiple similarity differences; determine among the multiple similarity differences A similarity difference that meets the first requirement; and a dynamic threshold corresponding to any row is determined based on the similarity difference that meets the first requirement.
  • the adjustment module is configured to adjust the similarity included in the k-th row of the initial similarity matrix that is smaller than the dynamic threshold corresponding to the k-th row to a first value, and The reference similarity matrix is obtained based on the adjustment results of each row, and k is a positive integer; or, among the similarities included in the k-th row of the initial similarity matrix, the similarity smaller than the dynamic threshold corresponding to the k-th row is compared with The second numerical values are multiplied and the reference similarity matrix is obtained based on the adjustment results of each row.
  • the determination module is configured to process the reference similarity matrix according to multiple reference parameters to obtain a similarity matrix corresponding to each reference parameter; according to the multiple reference parameters and The similarity matrix corresponding to each reference parameter determines the number of audio objects present in the multiple audio clips.
  • the determination module is configured to numerically adjust the reference similarity matrix for any one of the plurality of reference parameters according to the any reference parameter, to obtain The first similarity matrix, the numerical adjustment is used to simplify the reference similarity matrix; the first similarity matrix is symmetrized to obtain a second similarity matrix, and the second similarity matrix is located at the The similarity in row i and column j is the same as the similarity in row j and column i, and the i and j are positive integers not greater than the number of the plurality of audio clips; for the second similarity
  • the degree matrix is diffused in rows and columns to obtain a third similarity matrix, which is used to generate boundaries between multiple audio objects; the third similarity matrix is proportionally adjusted to obtain a fourth similarity matrix. , the proportion adjustment is used to adjust the similarities included in each row in the third similarity matrix within the same range; perform symmetry processing on the fourth similarity matrix to obtain the corresponding value of any reference parameter. Similarity matrix.
  • the determination module is configured to adjust the similarity of any reference parameter other than the similarity that meets the third requirement for the multiple similarities included in each row of the reference similarity matrix. is the third value, obtain the first similarity matrix; or, among the multiple similarities included in the reference similarity matrix, the similarities other than the similarities that meet the third requirement of any reference parameter are compared with The fourth numerical values are multiplied to obtain the first similarity matrix.
  • the determining module is used to determine the transposed matrix corresponding to the first similarity matrix; and transpose the first similarity matrix and the corresponding transposed matrix
  • the similarities located at the same position in the matrix are added to obtain the similarity matrix to be adjusted; the plurality of similarities included in the similarity matrix to be adjusted are halved to obtain the second similarity matrix.
  • the determination module is configured to determine the similarity between the i-th row and j-th column in the first similarity matrix and the similarity between the j-th row and j-th column in the first similarity matrix.
  • the maximum similarity among the similarities in the j-th row and i-th column is used as the maximum similarity between the i-th row and j-th column and the j-th row and i-th column in the second similarity matrix. similarity to obtain the second similarity matrix.
  • the determination module is used to determine the transpose matrix corresponding to the second similarity matrix; according to the second similarity matrix and the transpose corresponding to the second similarity matrix matrix, determine the third similarity matrix, the similarity located in the m-th row and the n-th column in the third similarity matrix is based on the similarity located in the m-th row in the second similarity matrix and the The similarity located in the nth column in the transposed matrix corresponding to the second similarity matrix is determined, and the m and n are positive integers that are not greater than the number of the multiple audio segments.
  • the determination module is configured to determine the maximum similarity corresponding to each row according to multiple similarities included in each row in the third similarity matrix; Multiple similarities included in each row are divided by the maximum similarity corresponding to each row to obtain the fourth similarity matrix.
  • the determining module is configured to determine The similarity matrix corresponding to the parameter determines the proportion value corresponding to each reference parameter, and the proportion value is used to indicate the number of similarities retained in the similarity matrix corresponding to the reference parameter; according to the similarity matrix corresponding to each reference parameter A ratio value that determines the number of audio objects present in the plurality of audio segments.
  • the determination module is configured to perform a Laplace transform on the similarity matrix corresponding to any one of the multiple reference parameters, to obtain The Laplacian matrix corresponding to any of the reference parameters; perform singular value decomposition on the Laplacian matrix to obtain multiple reference eigenvalues; determine the second eigenvalue and the third eigenvalue among the multiple reference eigenvalues.
  • a number of first characteristic values, the second characteristic value is the maximum value among the plurality of reference characteristic values, and the first characteristic value satisfies the requirement after sorting the plurality of reference characteristic values in a second order.
  • the second required reference feature value determine the difference between two adjacent first feature values in the first number of first feature values, and obtain a plurality of feature value differences; according to the second feature value , normalize the first eigenvalue difference to obtain the eigenvalue difference after normalization, where the first eigenvalue difference is the largest eigenvalue difference among the plurality of eigenvalue differences; According to the normalized feature value difference and any reference parameter, the proportion value corresponding to the any reference parameter is determined.
  • the determining module is configured to determine a first parameter among the plurality of reference parameters according to the proportion value corresponding to each reference parameter, and the first parameter is the plurality of reference parameters.
  • the corresponding reference parameter with the smallest proportion value is determined; multiple feature value differences corresponding to the first parameter are determined; and the first function is called to process the multiple feature value differences corresponding to the first parameter to obtain the result. Describes the number of audio objects present in multiple audio clips.
  • the clustering module is used to perform singular value decomposition on the similarity matrix corresponding to the first parameter to obtain multiple decomposition eigenvalues; determine among the multiple decomposition eigenvalues The decomposition eigenvalues correspond to the number of audio objects; determine the eigenvectors corresponding to the number of decomposition eigenvalues of the audio objects, and generate a decomposition matrix.
  • the number of rows of the decomposition matrix is the number of the audio objects, and the columns are The number is the number of the audio segments; according to the decomposition matrix, the feature vectors corresponding to the multiple audio segments are determined, and the feature vectors are used to indicate the corresponding audio segments; according to the number of the audio objects and the Feature vectors corresponding to multiple audio segments are clustered, and the audio segments are obtained by uttering sounds from each audio object.
  • inventions of the present application provide a computer device.
  • the computer device includes a processor and a memory. At least one program code is stored in the memory. The at least one program code is loaded and executed by the processor. , so that the computer device implements any of the above audio processing methods.
  • a computer-readable storage medium is also provided. At least one program code is stored in the computer-readable storage medium. The at least one program code is loaded and executed by the processor to enable the computer to implement any of the above.
  • the audio processing method is also provided.
  • a computer program or computer program product is also provided. At least one computer instruction is stored in the computer program or computer program product. The at least one computer instruction is loaded and executed by the processor to enable the computer to implement the above. Any audio processing method.
  • the initial similarity matrix is adjusted to obtain the reference similarity matrix.
  • the similarity of the voiceprint vectors of the audio clips of the same audio object can be brought closer.
  • Zooming out the similarity of the voiceprint vectors of audio clips of different audio objects makes the number of audio objects determined based on the reference similarity matrix more accurate; then based on the number of audio objects with higher accuracy, multiple audio clips are processed
  • Clustering is used to obtain the audio fragments corresponding to each audio object, so that the accuracy of the determined audio fragments corresponding to each audio object is higher, and the accuracy of audio object clustering is higher, which in turn can improve the audio processing effect of the audio fragments.
  • Figure 1 is a schematic diagram of the implementation environment of an audio processing method provided by an embodiment of the present application.
  • Figure 2 is a flow chart of an audio processing method provided by an embodiment of the present application.
  • Figure 3 is a schematic diagram of a similarity matrix determination process provided by an embodiment of the present application.
  • Figure 4 is a flow chart of another audio processing method provided by an embodiment of the present application.
  • Figure 5 is a schematic structural diagram of an audio processing device provided by an embodiment of the present application.
  • Figure 6 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • Figure 7 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • the audio processing method provided by the embodiment of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, games, etc.
  • Artificial Intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
  • artificial intelligence is a comprehensive technology of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence.
  • Artificial intelligence is the study of the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.
  • Machine Learning is a multi-field interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. and many other disciplines. Specializes in studying how computers can simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance.
  • Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications cover all fields of artificial intelligence.
  • Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, teaching learning and other technologies.
  • artificial intelligence technology has been researched and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless driving, autonomous driving, and drones. , robots, smart medical care, smart customer service, Internet of Vehicles, autonomous driving, smart transportation, etc.
  • Figure 1 is a schematic diagram of an implementation environment of an audio processing method provided by an embodiment of the present application. As shown in Figure 1, the implementation environment includes: a terminal device 101 and a server 102.
  • the audio processing method provided by the embodiment of the present application can be executed by the terminal device 101, or by the server 102, or by both the terminal device 101 and the server 102. This is not limited by the embodiment of the present application.
  • the server 102 undertakes the main calculation work and the terminal device 101 undertakes the secondary calculation work; or the server 102 undertakes the secondary calculation work and the terminal device 101 undertakes the main computing work; alternatively, the server 102 and the terminal device 101 adopt a distributed computing architecture for collaborative computing.
  • the terminal device 101 can be any electronic product that can perform human-computer interaction with the user through one or more methods such as keyboard, touch pad, touch screen, remote control, voice interaction or handwriting device.
  • Terminal devices 101 include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc.
  • the server 102 is a server, or a server cluster composed of multiple servers, or a cloud computing platform and a virtualization center, or any of the nodes in the blockchain system. This embodiment of the present application does not mention this. limited.
  • the server 102 communicates with the terminal device 101 through a wired network or a wireless network.
  • the server 102 has a data receiving function, a data processing function and a data sending function.
  • the server 102 may also have other functions, which are not limited in the embodiments of this application.
  • terminal equipment 101 and server 102 are only examples.
  • the embodiment of the present application provides an audio processing method, which is executed by a computer device.
  • the method can be applied to the implementation environment shown in Figure 1.
  • the computer device can be the terminal device 101 in Figure 1, or it can be the terminal device 101 in Figure 1.
  • the server 102 in this application is not limited in this embodiment. Taking the flow chart of an audio processing method provided by the embodiment of the present application shown in Figure 2 as an example, as shown in Figure 2, the method includes the following steps 201 to 205.
  • step 201 voiceprint vectors corresponding to multiple audio segments are determined, and the voiceprint vectors are used to indicate voiceprint features corresponding to the audio segments.
  • multiple audio segments need to be obtained first.
  • the multiple audio segments are at least two audio segments, and each audio segment corresponds to an audio object.
  • the audio object is the sound object of the audio clip, where the audio objects corresponding to different audio clips are the same or different.
  • the embodiment of the present application does not limit the acquisition process of multiple audio clips. For example, multiple candidate segments are stored in the storage space of the computer device, and multiple audio segments are obtained from the multiple candidate segments.
  • multiple voice segments may be used as audio segments to be processed, or a part of the voice segments may be selected as audio segments to be processed.
  • the song data is obtained as voice data, and the voice data is divided into multiple voice segments as audio segments according to the lyrics segmentation.
  • the corresponding durations of multiple audio clips may be the same or different, and this is not limited in the embodiments of the present application.
  • the corresponding duration of multiple audio clips is 2 seconds.
  • the corresponding duration of some audio clips among the multiple audio clips is 2 seconds, and the corresponding duration of some audio clips is 5 seconds.
  • the voiceprint vector corresponding to each audio segment is determined.
  • the features corresponding to each audio segment may be the MFCC (Mel-scaleFrequency Cepstral Coefficients, Mel-scale Frequency Cepstral Coefficients) corresponding to each audio segment, or the Mel-spectrum features corresponding to each audio segment, or other features. , the embodiment of the present application does not limit this.
  • the process of determining the voiceprint vector corresponding to each audio segment according to the characteristics corresponding to each audio segment includes: inputting the characteristics corresponding to the audio segment into the voiceprint extraction model, and inputting the results output by the voiceprint extraction model.
  • the voiceprint extraction model can be any model, which is not limited in the embodiments of this application.
  • the voiceprint extraction model can be a CLDNN (Convolution-Longshort-Term Mermony-Fully-Connected Deep Neural Networks, convolution-long short-term memory-fully connected neural network) model, or it can be based on TDNN (Time Delay Neural Network) , time-delay neural network) X-vector (the mainstream baseline model framework in the field of voiceprint recognition), or ecapa-tdnn (a model that extracts global features of speech).
  • CLDNN Convolution-Longshort-Term Mermony-Fully-Connected Deep Neural Networks, convolution-long short-term memory-fully connected neural network
  • TDNN Time Delay Neural Network
  • X-vector the mainstream baseline model framework in the field of voiceprint recognition
  • ecapa-tdnn a model that extracts global features of speech
  • an initial similarity matrix is determined based on the voiceprint vectors corresponding to each audio segment.
  • the initial similarity matrix includes the similarity between the voiceprint vectors corresponding to any two audio segments.
  • the voiceprint vector corresponding to each audio segment is determined in the above step 201, and the similarity between the voiceprint vectors corresponding to any two audio segments is determined based on the voiceprint vector corresponding to each audio segment. degree to obtain the initial similarity matrix.
  • the similarity between the voiceprint vectors corresponding to any two audio segments is determined according to the following formula (1).
  • a i, j d(v i , v j ) i, j ⁇ [1, N]
  • a i,j is the similarity between the voiceprint vector corresponding to the i-th audio segment and the voiceprint vector corresponding to the j-th audio segment
  • d( vi , vj ) is the distance
  • vi is the voiceprint vector corresponding to the i-th audio clip
  • v j is the voiceprint vector corresponding to the j-th audio clip
  • N is the total number of multiple audio clips.
  • the cosine similarity distance between the voiceprint vectors corresponding to any two audio clips can be used as the similarity between the voiceprint vectors corresponding to any two audio clips.
  • the similarity between the voiceprint vectors corresponding to any two audio clips can also be determined in other ways, which is not limited in the embodiments of the present application.
  • the number of rows of the initial similarity matrix is the number of multiple audio clips, and the number of columns is the number of multiple audio clips.
  • the initial similarity matrix is a symmetry matrix, that is, the similarity located in the i-th row and j-th column in the initial similarity matrix is the same as the similarity located in the j-th row and i-th column.
  • i and j are positive integers not greater than the number of multiple audio clips.
  • any two audio clips corresponding to any two audio clips are the same audio object.
  • any two audio clips corresponding to The the lower the similarity between voiceprint vectors, the lower the possibility that the audio objects corresponding to any two audio clips are the same audio object.
  • the similarity between the voiceprint vectors of any two audio clips is determined according to the above formula (1). Then, based on the similarity between the voiceprint vectors of any two audio clips, the initial similarity matrix is determined to be a 5*5 matrix.
  • the initial similarity matrix looks like this:
  • a 1, 1 is used to represent the similarity between the voiceprint vector corresponding to the first audio clip and the voiceprint vector corresponding to the first audio clip
  • a 1, 2 is used to represent the similarity between the voiceprint vector corresponding to the first audio clip and the voiceprint vector corresponding to the first audio clip.
  • the meanings represented by other elements in the initial similarity matrix are the same as those represented by a 1,1 and a 1,2 have similar meanings and will not be described in detail here.
  • step 203 the initial similarity matrix is adjusted according to the dynamic threshold corresponding to each row in the initial similarity matrix to obtain a reference similarity matrix.
  • the dynamic threshold is used to adjust the similarity difference between different similarities.
  • the dynamic threshold is used to bring the difference between the similarities of the voiceprint vectors of the audio clips of the same audio object closer, and/or to make the difference between the similarities of the voiceprint vectors of the audio clips of different audio objects farther away.
  • bringing the difference between the similarities of the voiceprint vectors of the audio clips of the same audio object closer refers to the difference between the first similarity and the second similarity
  • the first similarity is The similarity between the voiceprint vector of the first audio segment and the voiceprint vector of the second audio segment
  • the second similarity is the similarity between the voiceprint vector of the first audio segment and the voiceprint vector of the third audio segment
  • the first audio segment, the second audio segment, and the third audio segment correspond to the same audio object.
  • the difference between the similarities between the voiceprint vectors of the audio clips of different audio objects refers to the difference between the first similarity and the third similarity.
  • the third similarity is the sound of the first audio clip.
  • the similarity between the voiceprint vector and the voiceprint vector of the fourth audio segment, the first audio segment and the fourth audio segment correspond to different audio objects.
  • the initial similarity matrix is adjusted according to the dynamic threshold corresponding to each row in the initial similarity matrix.
  • the dynamic threshold corresponding to each row in the initial similarity matrix needs to be determined.
  • the process includes: for any row in the initial similarity matrix, sort the similarities in the preset similarity range among the multiple similarities included in any row in the first order to obtain the first sorting result; according to the first sorting As a result, the similarity difference between two adjacent similarities within the preset similarity range is determined, and multiple similarity differences are obtained. The number of similarity differences is smaller than the similarity included in any row.
  • the number of degrees that is, determine the similarity difference between two adjacent similarities in the first sorting result, and obtain multiple similarity differences; determine that the first requirement is met among the multiple similarity differences.
  • the similarity difference ; determine the dynamic threshold corresponding to any row based on the similarity difference that meets the first requirement.
  • the minuend corresponding to the similarity difference that meets the first requirement is used as the dynamic threshold corresponding to any row.
  • the first order may be an order from small to large, or may be an order from large to small, which is not limited in the embodiment of the present application.
  • the preset similarity range is set based on experience or adjusted according to the implementation environment, and is not limited in the embodiments of the present application. For example, the preset similarity range is [-1, 1].
  • the similarity difference that satisfies the first requirement among the multiple similarity differences refers to the largest similarity difference among the multiple similarity differences.
  • the first order is from small to large
  • the preset similarity range is [-1, 1]
  • the similarities included in any row of the initial similarity matrix are: 1, -0.3, 0.7, 0.5, 0.9.
  • the similarities within the preset similarity range are sorted from small to large, and the first sorting results obtained are: -0.3, 0.5, 0.7, 0.9, 1.
  • the largest similarity difference among multiple similarity differences is 0.8. Therefore, the minuend 0.5 corresponding to 0.8 is used as the dynamic threshold corresponding to any row. In some embodiments, when there are multiple minuends corresponding to the maximum similarity difference, any one of the minuends is determined as the dynamic threshold.
  • the similarity difference between two adjacent similarities within the preset similarity range is determined.
  • the similarity difference can also be determined based on multiple similarity differences.
  • Each similarity difference value determines a similarity difference vector, and the similarity difference vector includes multiple similarity difference values.
  • the following formula (2) is a similarity difference vector.
  • the similarity difference vector corresponding to any row of gap q , a′ q, 1 is the similarity that ranks first in the first sorting result after sorting from small to large
  • a′ q, 2 is the similarity that is ranked second in the first sorted result sorted from small to large
  • a′ q, 3 is the third ranked result in the first sorted result sorted from small to large
  • the similarity of , a′ q, N is the similarity at the last position in the first sorted result after sorting from small to large
  • a′ q, N-1 is the similarity of the last sorted result after sorting from small to large.
  • the second-to-last similarity in a sorted result is the similarity that ranks first in the first sorting result after sorting from small to large
  • a′ q, 2 is the similarity that is ranked second in the first sorted result sorted from small to large
  • a′ q, 3 is the third ranked result in the first sorted result sorted from
  • Implementation method 1 Among the similarities included in the k-th row of the initial similarity matrix, the similarity that is smaller than the dynamic threshold corresponding to the k-th row is adjusted to the first value, and the reference similarity matrix is obtained based on the adjustment results of each row, k is positive integer.
  • the first numerical value is set based on experience or adjusted according to the implementation environment, which is not limited in the embodiments of the present application.
  • the first value is 0.
  • the initial similarity matrix is Among them, the dynamic threshold corresponding to the first row is 0.5, the dynamic threshold corresponding to the second row is 0.6, the dynamic threshold corresponding to the third row is 0.7, the dynamic threshold corresponding to the fourth row is 0.2, and the dynamic threshold corresponding to the fifth row is 0.9 , then among the similarities included in each row of the initial similarity matrix, the similarity smaller than the dynamic threshold corresponding to each row is adjusted to 0, and the obtained reference similarity matrix is
  • Implementation method two among the similarities included in the k-th row of the initial similarity matrix, the similarities smaller than the dynamic threshold corresponding to the k-th row are multiplied by the second value, and the reference similarity matrix is obtained based on the adjustment results of each row.
  • the second numerical value is set based on experience or adjusted according to the implementation environment, which is not limited in the embodiments of the present application.
  • the second value is 0.01.
  • the initial similarity matrix is Among them, the dynamic threshold corresponding to the first row is 0.5, the dynamic threshold corresponding to the second row is 0.6, the dynamic threshold corresponding to the third row is 0.7, the dynamic threshold corresponding to the fourth row is 0.2, and the dynamic threshold corresponding to the fifth row is 0.9 , then among the similarities included in each row of the initial similarity matrix, the similarity smaller than the dynamic threshold corresponding to each row is multiplied by 0.01, and the resulting reference similarity matrix is
  • the distance between the first similarity and the second similarity in the reference similarity matrix is smaller than the distance between the first similarity and the second similarity in the initial similarity matrix.
  • the distance between the similarity and the third similarity is greater than the distance between the first similarity and the third similarity in the initial similarity matrix, so as to achieve the similarity between the voiceprint vectors of the audio fragments of the same audio object.
  • the difference is the difference between the voiceprint vectors of audio clips of different audio objects.
  • step 204 the number of audio objects that produce multiple audio segments is determined according to the reference similarity matrix.
  • the process of determining the number of audio objects present in multiple audio clips based on the reference similarity matrix includes: processing the reference similarity matrix according to multiple reference parameters to obtain the corresponding number of each reference parameter. Similarity matrix; determine the number of audio objects existing in multiple audio clips based on multiple reference parameters and the similarity matrix corresponding to each reference parameter.
  • the reference parameters are set based on experience or adjusted according to the implementation environment, which is not limited in the embodiments of the present application. The number of reference parameters is not limited in this application.
  • the reference similarity matrix is processed according to multiple reference parameters to obtain the similarity matrix corresponding to each reference parameter.
  • the process of obtaining the similarity matrix corresponding to each reference parameter is similar.
  • only any one of the multiple reference parameters corresponds to the reference parameter.
  • the determination process of the similarity matrix is described as an example. The process includes the following steps 1 to 5.
  • Step 1 According to any reference parameter, perform numerical adjustment on the reference similarity matrix to obtain the first similarity matrix.
  • the numerical adjustment is used to simplify the reference similarity matrix.
  • any reference parameter there are the following two ways to numerically adjust the reference similarity matrix to obtain the first similarity matrix.
  • Method 1 For the multiple similarities included in each row of the reference similarity matrix, adjust the similarities other than the similarity that meets the third requirement of any reference parameter to a third value to obtain the first similarity matrix.
  • the third numerical value is set based on experience or adjusted according to the implementation environment, which is not limited in the embodiments of the present application.
  • the third value is 0.
  • the similarity of any reference parameter that meets the third requirement refers to the maximum similarity of any reference parameter, that is, the number of similarities corresponding to any reference parameter is adjusted to the third value and is adjusted to the third value.
  • the similarity of is the maximum similarity in the row of the reference similarity matrix.
  • sorting results corresponding to each row except for the previous reference parameter similarity
  • the similarities outside are adjusted to the third value to obtain the first similarity matrix.
  • any reference parameter is 3, the third value is 0, and the reference similarity matrix is According to any reference parameter, it is determined that the three similarities that meet the third requirement in the first row are 1, 0.9, and 0.7, the three similarities that meet the third requirement in the second row are 1, 0.6, and -0.003, and the three similarities that meet the third requirement in the second row are 1, 0.6, and -0.003.
  • the three similarities that meet the third requirement in the row are 1, 0.8, and 0.7.
  • the three similarities that meet the third requirement in the fourth row are 1, 0.8, and 0.5.
  • the three similarities that meet the third requirement in the fifth row are 1, 0.9, and 0.006. Performed on the reference similarity matrix Adjust to get the first similarity matrix as
  • Method 2 Multiply the similarities among the multiple similarities included in the reference similarity matrix, except the similarities that meet the third requirement for any reference parameter, by the fourth value to obtain the first similarity matrix.
  • the fourth numerical value is set based on experience or adjusted according to the implementation environment, which is not limited in the embodiments of the present application.
  • the fourth value is 0.01.
  • Multiply the similarities other than the number of similarities with the fourth value to obtain the first similarity matrix that is, multiply the similarities with the number of similarities other than the number corresponding to any reference parameter by the fourth value, and multiply the number of similarities with the number corresponding to any reference parameter.
  • the similarity is the maximum similarity in the row of the reference similarity matrix.
  • any reference parameter is 3, the fourth value is 0.01, and the reference similarity matrix is According to any reference parameter, it is determined that the three similarities that meet the third requirement in the first row are 1, 0.9, and 0.7, the three similarities that meet the third requirement in the second row are 1, 0.6, and -0.003, and the three similarities that meet the third requirement in the second row are 1, 0.6, and -0.003.
  • the three similarities that meet the third requirement in the row are 1, 0.8, and 0.7.
  • the three similarities that meet the third requirement in the fourth row are 1, 0.8, and 0.5.
  • the three similarities that meet the third requirement in the fifth row are are 1, 0.9, and 0.006. Adjust the reference similarity matrix to obtain the first similarity matrix as the reference similarity matrix as
  • any of the above methods can be selected to numerically adjust the reference similarity matrix to obtain the first similarity matrix, which is not limited in the embodiments of the present application.
  • B is the first similarity matrix
  • A is the reference similarity matrix
  • p is any reference parameter
  • Threshold is the numerical adjustment function.
  • Step 2 Perform symmetry processing on the first similarity matrix to obtain a second similarity matrix.
  • the similarity located in the i-th row and j-th column is the same as the similarity located in the j-th row and i-th column.
  • i and j are positive integers not greater than the number of multiple audio clips.
  • the first similarity matrix after dynamic threshold processing and numerical adjustment may be an asymmetric matrix, so the first similarity matrix is symmetrized. Due to the similarity between the voiceprint vector of the i-th audio segment and the voiceprint vector of the j-th audio segment, the similarity between the voiceprint vector of the j-th audio segment and the voiceprint vector of the i-th audio segment are the same, that is, the similarity located in the i-th row and j-th column is the same as the similarity located in the j-th row and i-th column.
  • the first similarity matrix it is necessary to symmetrize the first similarity matrix so that the similarity in the i-th row and j-th column is the same.
  • the similarity in row i and column j is the same as the similarity in row j and column i.
  • there are at least the following ways to perform the first similarity matrix Perform row symmetry processing to obtain the second similarity matrix.
  • Method 1 Determine the transposed matrix corresponding to the first similarity matrix; add the similarities located at the same position in the first similarity matrix and the transposed matrix corresponding to the first similarity matrix to obtain the similarity matrix to be adjusted; treat Adjust multiple similarities included in the similarity matrix and perform a half operation to obtain a second similarity matrix.
  • C is the second similarity matrix
  • B is the first similarity matrix
  • B T is the transposed matrix corresponding to the first similarity matrix
  • the first similarity matrix is The transposed matrix corresponding to the first similarity matrix is Add the similarities at the same position in the first similarity matrix and the transposed matrix corresponding to the first similarity matrix, and the obtained similarity matrix to be adjusted is: Perform a half operation on multiple similarities included in the similarity matrix to be adjusted, and the second similarity matrix obtained is
  • Method 1 is a process of determining the second similarity matrix by averaging based on the first similarity matrix and the transposed matrix corresponding to the first similarity matrix.
  • Method 2 Determine the greatest similarity between the similarity located in the i-th row and j-th column in the first similarity matrix and the similarity located in the j-th row and i-th column in the first similarity matrix, and use the largest similarity as The similarity between the i-th row and j-th column and the j-th row and i-th column in the second similarity matrix is used to obtain the second similarity matrix.
  • a′ i,j is the similarity located in the i-th row and j-th column in the second similarity matrix
  • a ij is the similarity located in the i-th row and j-th column in the first similarity matrix
  • a ji is the similarity located in the j-th row and i-th column in the first similarity matrix.
  • the first similarity matrix is Then the second similarity matrix is
  • Method 2 is a process of determining the second similarity matrix by taking the maximum value based on the first similarity matrix.
  • any of the above methods can be selected to perform symmetry processing on the first similarity matrix to obtain the second similarity matrix, which is not limited in the embodiments of the present application.
  • Step 3 Perform row-column diffusion on the second similarity matrix to obtain a third similarity matrix.
  • the third similarity matrix is used to generate boundaries between multiple audio objects.
  • the process of performing row-column diffusion on the second similarity matrix to obtain the third similarity matrix includes: determining the transpose matrix corresponding to the second similarity matrix, and based on the second similarity matrix and the second similarity matrix.
  • the transpose matrix corresponding to the similarity matrix determines the third similarity matrix.
  • the similarity in the m-th row and n-th column in the third similarity matrix is based on the similarity in the m-th row and the second similarity in the second similarity matrix.
  • the similarity in the nth column of the transposed matrix corresponding to the degree matrix is determined, and m and n are positive integers not greater than the number of multiple audio clips.
  • the similarity located at the m-th row and the n-th column in the third similarity matrix are located at the similarity located at the m-th row in the second similarity matrix and the corresponding transposed matrix of the second similarity matrix are located at the The result of corresponding multiplication and addition of the similarity in the n columns is used as the similarity located in the m-th row and n-th column in the third similarity matrix.
  • the similarities located in the m-th row of the second similarity matrix are 1, 0, 0.7, 0.5, and 0.9 respectively, and the similarities located in the n-th column of the corresponding transposed matrix of the second similarity matrix are 1 respectively.
  • D is the third similarity matrix
  • C is the second similarity matrix
  • C T is the transposed matrix corresponding to the second similarity matrix
  • the second similarity matrix is The transposed matrix corresponding to the second similarity matrix is Then the third similarity matrix is
  • Step 4 Proportionally adjust the third similarity matrix to obtain a fourth similarity matrix.
  • the proportional adjustment is used to adjust the similarities included in each row of the third similarity matrix within the same range.
  • the process of proportionally adjusting the third similarity matrix to obtain the fourth similarity matrix includes: determining the maximum similarity corresponding to each row according to multiple similarities included in each row of the third similarity matrix. degree; divide multiple similarities included in each row of the third similarity matrix by the maximum similarity corresponding to each row to obtain a fourth similarity matrix.
  • the third similarity matrix is proportionally adjusted according to the following formula (7) to obtain a fourth similarity matrix.
  • a′′ ij is the similarity located in the i-th row and j-th column in the fourth similarity matrix
  • a ij is the similarity located in the i-th row and j-th column in the third similarity matrix
  • a ik is the maximum similarity corresponding to the i-th row in the third similarity matrix
  • k is the column of the maximum similarity corresponding to the i-th row in the third similarity matrix.
  • the third similarity matrix is Among them, the maximum similarity corresponding to the first row is 2.55, the maximum value corresponding to the second row is 1.36, the maximum value corresponding to the third row is 2.13, the maximum value corresponding to the fourth row is 1.95, and the maximum value corresponding to the fifth row is 2.17. According to the maximum similarity corresponding to each row, the third similarity matrix is proportionally adjusted, and the fourth similarity matrix obtained is
  • Step 5 Perform symmetry processing on the fourth similarity matrix to obtain the similarity matrix corresponding to any reference parameter.
  • the process of symmetrizing the fourth similarity matrix to obtain the similarity matrix corresponding to any reference parameter is the same as the above process of symmetrizing the first similarity matrix to obtain the second similarity.
  • the matrix process is similar and will not be repeated here.
  • step 1 the similarity matrix corresponding to each reference parameter is determined.
  • Figure 3 is a schematic diagram of a similarity matrix determination process provided by an embodiment of the present application.
  • (1) in Figure 3 is the initial similarity matrix
  • (2) in Figure 3 is the reference similarity matrix
  • (3) in Figure 3 is the first similarity matrix
  • (4) in Figure 3 is the third Similarity matrix
  • (5) in Figure 3 is the fourth similarity matrix
  • (6) in Figure 3 is the similarity matrix corresponding to the reference parameters.
  • the horizontal axis in (1) in Figure 3 is the number of audio clips
  • the vertical axis is the number of audio clips. The higher the brightness of the area in Figure 3, the higher the similarity between the voiceprint vectors of the two audio clips. .
  • the process of determining the number of audio objects present in multiple audio clips based on multiple reference parameters and the similarity matrix corresponding to each reference parameter includes: based on the multiple reference parameters and the corresponding similarity matrix of each reference parameter. similarity matrix, determine the proportion value corresponding to each reference parameter, the proportion value is used to indicate the number of similarities retained in the similarity matrix corresponding to the reference parameter; according to the proportion value corresponding to each reference parameter, determine the presence of multiple audio clips the sound The number of frequency objects.
  • the process of determining the proportion value corresponding to each reference parameter includes: for any reference parameter among the multiple reference parameters, for any reference parameter corresponding Perform Laplace transformation on the similarity matrix to obtain the Laplace matrix corresponding to any reference parameter; perform singular value decomposition on the Laplace matrix to obtain multiple reference eigenvalues; determine the number of reference eigenvalues among the multiple reference eigenvalues. Two eigenvalues and a first number of first eigenvalues, the second eigenvalue is the maximum value among multiple reference eigenvalues, and the first eigenvalue satisfies the second requirement after sorting the multiple reference eigenvalues in a second order. reference characteristic value.
  • the first eigenvalue difference is the largest eigenvalue difference among multiple eigenvalue differences; according to the normalized eigenvalue difference and any reference parameter, Determine the proportional value corresponding to any reference parameter.
  • the first number is set based on experience or adjusted according to the implementation environment, which is not limited in the embodiments of the present application. For example, the first quantity is 3.
  • the second order may be an order from small to large, or may be an order from large to small, which is not limited in the embodiment of the present application.
  • the first feature value is the first number of reference feature values after sorting the multiple reference feature values in order from small to large.
  • the first feature value is the first number of reference feature values after sorting the plurality of reference feature values in order from large to small.
  • any one of the first eigenvalues can be determined based on the multiple eigenvalue differences.
  • the eigenvalue difference vector includes multiple eigenvalue differences.
  • the following formula (8) is the feature value difference vector corresponding to any reference parameter.
  • e p is the eigenvalue difference vector corresponding to any reference parameter
  • ⁇ p, 1 is the reference feature that ranks first after sorting multiple reference eigenvalues in order from small to large.
  • Value ⁇ p, 2 is the second reference eigenvalue after sorting multiple reference eigenvalues in order from small to large
  • ⁇ p, 3 is sorting multiple reference eigenvalues in order from small to large
  • ⁇ p, Y is the reference eigenvalue located in the Yth position after sorting multiple reference eigenvalues in order from small to large
  • ⁇ p, Y-1 is the reference eigenvalue in order from small to large
  • Y is the first quantity.
  • the eigenvalue difference is normalized according to the following formula (9) to obtain the normalized eigenvalue difference.
  • g P is the eigenvalue difference after normalization
  • max(e p ) is the first eigenvalue difference
  • ⁇ max is the second eigenvalue
  • ⁇ is the normalization parameter
  • ⁇ The value is 1*10 -10 .
  • the proportion value corresponding to any reference parameter is determined according to the following formula (10).
  • r(p) is the proportion value corresponding to any reference parameter
  • p is any reference parameter
  • g p is the eigenvalue difference after normalization.
  • the process of determining the number of audio objects present in multiple audio clips according to the proportion value corresponding to each reference parameter includes: determining the number of audio objects present in the multiple reference parameters according to the proportion value corresponding to each reference parameter.
  • the first parameter is the reference parameter with the smallest corresponding proportion value among the multiple reference parameters; determine the multiple reference parameters corresponding to the first parameter.
  • characteristic value differences calling the first function to process the plurality of characteristic value differences corresponding to the first parameter, and obtaining the number of audio objects existing in the plurality of audio clips.
  • the reference parameter corresponding to the smallest proportion value is used as the first parameter.
  • the process of determining the differences between multiple eigenvalues corresponding to the first parameter is: determining the similarity matrix corresponding to the first parameter, performing Laplace transform on the similarity matrix corresponding to the first parameter, and obtaining the first
  • the Laplacian matrix corresponding to the parameter perform singular value decomposition on the Laplacian matrix corresponding to the first parameter, obtain multiple reference eigenvalues, and sort the smallest first number of reference eigenvalues among the multiple reference eigenvalues. , using the difference between two adjacent reference feature values in the sorting as the multiple feature value differences corresponding to the first parameter.
  • the process of calling the first function to process multiple feature value differences corresponding to the first parameter, and obtaining the number of audio objects present in the multiple audio clips includes: combining the multiple feature value differences corresponding to the first parameter into a first
  • the feature value difference vector corresponding to the parameter is called the first function to process the feature value difference vector corresponding to the first parameter to obtain the number of audio objects existing in the multiple audio clips.
  • the plurality of feature value differences corresponding to the first parameter are processed according to the following formula (11) to obtain the number of audio objects present in the plurality of audio clips.
  • M is the number of audio objects existing in multiple audio clips
  • arg max() is the first function
  • the eigenvalue difference vector corresponding to the first parameter is a vector composed of multiple eigenvalue differences corresponding to the first parameter.
  • the similarity matrix corresponding to the first parameter is the matrix Q, perform Laplace transform on the matrix Q, and obtain the Laplace matrix P corresponding to the first parameter, and perform singular value decomposition on the matrix P to obtain multiple Reference feature values (respectively a, b, c, d, e, f), sort multiple reference feature values in order from small to large, and obtain the sorting result (b, c, a, e, f, d) , the first number is 3, and the difference between the two adjacent reference feature values among the three smallest reference values in the sorting result is used as the multiple feature value differences corresponding to the first parameter, and the multiple feature value differences are c-b, a-c respectively. Therefore, the vector composed of c-b and a-c is used as the eigenvalue difference vector, that is, the eigenvalue difference vector is [c-b, a-c].
  • step 205 multiple audio segments are clustered according to the number of audio objects to obtain audio segments resulting from the sounds of each audio object.
  • the multiple audio segments are clustered according to the number of audio objects to obtain the audio corresponding to each audio object.
  • the process of the fragment includes: performing singular value decomposition on the similarity matrix corresponding to the first parameter to obtain multiple decomposition eigenvalues; determining the number of decomposition eigenvalues of the audio object among the multiple decomposition eigenvalues; determining the number of decompositions of the audio object.
  • the eigenvectors corresponding to the eigenvalues respectively; decompose the eigenvectors corresponding to the eigenvalues according to the number of audio objects to generate a decomposition matrix.
  • the number of rows of the decomposition matrix is the number of audio objects and the number of columns is the number of audio clips; according to the decomposition matrix , determine the feature vectors corresponding to multiple audio clips, and the feature vectors are used to indicate the corresponding audio clips; according to the number of audio objects and the feature vectors corresponding to the multiple audio clips, cluster the multiple audio clips to obtain each audio The audio clip corresponding to the object.
  • the determined decomposition feature value is the smallest number of decomposition feature values of the audio object.
  • the smallest three decomposition feature values are determined among the multiple decomposition feature values. Determine the eigenvectors corresponding to these three decomposition eigenvalues.
  • the eigenvectors corresponding to the decomposition eigenvalues are 1*5 eigenvectors.
  • Three 1*5 eigenvectors are formed into a 3*5 decomposition matrix.
  • the first column in the decomposition matrix is used as the feature vector corresponding to the first audio clip
  • the second column is used as the feature vector corresponding to the second audio clip
  • the third column is used as the feature vector corresponding to the third audio clip
  • the fourth column is used as The feature vector corresponding to the fourth audio clip
  • the fifth column is the feature vector corresponding to the fifth audio clip.
  • the eigenvectors corresponding to the three decomposed eigenvalues are [x 1 , x 2 , x 3 , x 4 , x 5 ], [y 1 , y 2 , y 3 , y 4 , y 5 ], [ z 1 , z 2 , z 3 , z 4 , z 5 ], then the decomposition matrix composed of the eigenvectors corresponding to the three decomposition eigenvalues is: Therefore, take [x 1 , y 1 , z 1 ] as the feature vector corresponding to the first audio clip, take [x 2 , y 2 , z 2 ] as the feature vector corresponding to the second audio clip, and take [x 3 , y 3 , z 3 ] as the feature vector corresponding to the third audio segment, [x 4 , y 4 , z 4 ] as the feature vector corresponding to the fourth audio segment, and [x 5 , y 5 ,
  • the multiple audio segments are clustered through the K-means (K-means) clustering algorithm to obtain the audio segments corresponding to each audio object,
  • K-means K-means
  • the value of K is the number of audio objects.
  • other clustering algorithms can also be used to cluster multiple audio clips, which is not limited in the embodiments of the present application.
  • audio clip 1 there are five audio clips to be processed, namely audio clip 1, audio clip 2, audio clip 3, audio clip 4 and audio clip 5.
  • the number of audio objects existing in the audio clip to be processed is 3.
  • the audio fragments corresponding to audio object 1 are audio fragment 1 and audio fragment 3
  • the audio fragments corresponding to audio object 2 are audio fragment 5
  • the audio fragments corresponding to audio object 3 are audio fragment 2 and audio fragment 4. .
  • the multiple audio clips can also be determined based on the number of audio objects and the voiceprint vectors corresponding to the multiple audio clips.
  • the audio clips are clustered to obtain the audio clips corresponding to each audio object.
  • the process of clustering multiple audio segments according to the number of audio objects and the voiceprint vectors corresponding to the multiple audio segments to obtain the audio segments corresponding to each audio object is the same as the process described above based on the number of audio objects and the multiple audio segments.
  • the process of clustering multiple audio clips using corresponding feature vectors to obtain the audio clips corresponding to each audio object is similar and will not be described again here.
  • the audio processing method provided by the embodiment of the present application can be applied in the field of games to determine whether the same game account (or the same smart device) is used by several users.
  • collect the audio clips of the user when using the game account (or the smart device) call the audio processing method provided by the embodiment of the present application, determine the voiceprint vector corresponding to each audio clip, and determine the voiceprint vector corresponding to each audio clip according to the sound pattern corresponding to each audio clip.
  • the texture vector is used to determine the number of audio objects present in the audio clip and the audio clip corresponding to each audio object, so as to know how many users the game account (or the smart device) can use.
  • the above method adjusts the initial similarity matrix according to the dynamic threshold corresponding to each row in the initial similarity matrix, and then obtains the reference similarity matrix.
  • the similarity of the voiceprint vectors of audio clips of the same audio object can be brought closer degree, zoom out the similarity of the voiceprint vectors of audio fragments of different audio objects, so that the number of audio objects determined based on the reference similarity matrix is more accurate; and then based on the number of audio objects with higher accuracy, multiple audio
  • the fragments are clustered to obtain the audio fragments corresponding to each audio object, so that the accuracy of the determined audio fragments corresponding to each audio object is higher, and the accuracy of audio object clustering is higher, which can improve the audio processing effect of the audio fragments.
  • the method provided in this embodiment is to sort the person rows in the initial similarity matrix by similarity, thereby determining the dynamic threshold based on the difference between adjacent similarities in the similarity sorting, so as to use the dynamic threshold to determine the initial similarity.
  • the matrix is adjusted to improve the accuracy of the similarity of voiceprints that belong to the same audio object and the similarity of voiceprints of different audio objects that are far away, and the accuracy of determining the number of audio objects is improved.
  • the method provided in this embodiment sets a first numerical value for the similarity in the initial similarity matrix based on a dynamic threshold, which improves the adjustment efficiency of the initial similarity matrix.
  • the method provided in this embodiment operates and sets the similarity and the second value in the initial similarity matrix based on a dynamic threshold, which improves the setting flexibility and accuracy of the adjusted reference similarity matrix.
  • the method provided in this embodiment generates multiple reference parameters, processes the reference similarity matrix based on the multiple reference parameters, and obtains the similarity matrix corresponding to each reference parameter, thereby improving the accuracy of the similarity matrix and improving the efficiency of the similarity matrix. Accuracy in determining the number of audio objects.
  • the method provided in this embodiment adjusts the reference similarity matrix when determining the similarity matrix corresponding to the reference parameter. After the adjustment, since the adjusted first similarity matrix may be asymmetric, the first similarity matrix is symmetrized, and then column diffusion and proportion adjustment are performed, which improves the accuracy of determining the similarity matrix.
  • the method provided in this embodiment determines any reference parameter similarity according to the third requirement when determining the first similarity matrix, and numerically adjusts other similarities, and uses the first similarity matrix and the corresponding transpose The matrix determines the second similarity matrix, which improves the accuracy of the similarity matrix corresponding to the reference parameter.
  • Figure 4 is a flow chart of another audio processing method provided by an embodiment of the present application. As shown in Figure 4, the method includes the following steps 401 to 415.
  • step 201 this process has been described in step 201 above, and will not be described again here.
  • the signal preprocessing method for the audio clip includes at least one of segmentation, noise reduction, sampling, quantization and other processing methods.
  • the voiceprint extraction model can be a CLDNN model, an X-vector based on TDNN, or ecapa-tdnn.
  • step 202 this process has been described in step 202 above, and will not be described again here.
  • the similarity between the voiceprint vectors corresponding to any two audio clips is determined, and an initial similarity matrix is constructed, in which the number of vertical and horizontal directions of the initial similarity matrix corresponds to the number of audio clips, thereby constructing each audio clip
  • the correspondence matrix of the similarity between the voiceprint vectors is used as the initial similarity matrix.
  • step 203 this process has been described in step 203 above, and will not be described again here.
  • the dynamic threshold is used to make the similarity between the voiceprint features of audio clips corresponding to the same speaking object closer, or the dynamic threshold is used to make the similarity farther between the voiceprint features of the audio clips corresponding to different speaking objects.
  • any similarity other than the similarity that meets the third requirement of any reference parameter is adjusted to a third value to obtain the first similarity matrix; or, Among the multiple similarities included in the reference similarity matrix, the similarities other than the similarities that meet the third requirement of any reference parameter are multiplied by the fourth numerical value to obtain the first similarity matrix.
  • this process has been described in step 204 above, and will not be described again here. Since the first similarity matrix is asymmetric when the reference similarity matrix is adjusted to the first similarity matrix, the first similarity matrix is symmetrized to obtain the second similarity matrix.
  • this process has been described in step 204 above, and will not be described again here.
  • the second similarity matrix is diffused in rows and columns through the transposed matrix to obtain the third transposed matrix.
  • step 204 determine the maximum similarity corresponding to each row according to the multiple similarities included in each row in the third similarity matrix; compare the multiple similarities included in each row in the third similarity matrix with the maximum similarity corresponding to each row respectively. Divide to obtain the fourth similarity matrix.
  • step 204 Since the fourth similarity matrix is asymmetric after the proportion adjustment of the third similarity matrix, the fourth similarity matrix is symmetrized to obtain a similarity matrix corresponding to the reference parameters.
  • the scale value is used to indicate the number of similarities retained in the similarity matrix corresponding to the reference parameter.
  • the smaller the ratio value the smaller the number of similarities retained in the similarity matrix corresponding to the reference parameter, and the higher the accuracy of the number of audio objects subsequently determined; conversely, the larger the ratio value, the smaller the number of similarities retained in the similarity matrix corresponding to the reference parameter.
  • the greater the number of similarities retained in the matrix the less accurate the subsequent determination of the number of audio objects.
  • step 204 this process has been described in step 204 above, and will not be described again here.
  • step 204 this process has been described in step 204 above, and will not be described again here.
  • step 205 this process has been described in step 205 above, and will not be described again here.
  • step 205 this process has been described in step 205 above, and will not be described again here.
  • Figure 5 shows a schematic structural diagram of an audio processing device provided by an embodiment of the present application. As shown in Figure 5, the device includes:
  • the determination module 501 is used to determine the voiceprint vectors corresponding to multiple audio clips, and the voiceprint vector is used to indicate the voiceprint characteristics corresponding to the audio clips;
  • the determination module 501 is also used to determine an initial similarity matrix based on the voiceprint vector corresponding to each audio clip.
  • the initial similarity matrix includes the similarity between the voiceprint vectors corresponding to any two audio clips;
  • the adjustment module 502 is used to adjust the initial similarity matrix according to the dynamic threshold corresponding to each row in the initial similarity matrix to obtain a reference similarity matrix.
  • the dynamic threshold is used to adjust the similarity difference between different similarities. adjust;
  • the determination module 501 is also configured to determine the number of audio objects that sound out and obtain the plurality of audio segments according to the reference similarity matrix;
  • the clustering module 503 is configured to cluster the plurality of audio segments according to the number of the audio objects to obtain the audio segments produced by the sounds of each audio object.
  • the determination module 501 is also configured to, for any row in the initial similarity matrix, sort the similarities in the any row that are within the preset similarity range in the first order. , obtain the first sorting result; determine the similarity difference between two adjacent similarities in the first sorting result, and obtain multiple similarity differences; determine among the multiple similarity differences that satisfy the first The required similarity difference; determine the dynamic threshold corresponding to any row based on the similarity difference that meets the first requirement.
  • the adjustment module 502 is configured to adjust the similarity included in the k-th row of the initial similarity matrix that is less than the dynamic threshold corresponding to the k-th row to a first value, and based on The adjustment results for each row are In the reference similarity matrix, k is a positive integer; or, among the similarities included in the k-th row of the initial similarity matrix, the similarity smaller than the dynamic threshold corresponding to the k-th row is multiplied by the second value, And based on the adjustment results of each row, the reference similarity matrix is obtained.
  • the determination module 501 is used to process the reference similarity matrix according to multiple reference parameters to obtain the similarity matrix corresponding to each reference parameter; according to the multiple reference parameters and the similarity matrix corresponding to each reference parameter Similarity matrix that determines the number of audio objects present in multiple audio clips.
  • the determination module 501 is configured to perform numerical adjustment on the reference similarity matrix according to any reference parameter among the plurality of reference parameters to obtain a first similarity matrix, with a numerical value Adjustment is used to simplify the reference similarity matrix; symmetrize the first similarity matrix to obtain the second similarity matrix.
  • the similarity in the second similarity matrix located in the i-th row and j-th column is the same as the similarity located in the j-th row and i
  • the similarity of the columns is the same, i and j are positive integers not greater than the number of multiple audio clips; perform row-column diffusion on the second similarity matrix to obtain the third similarity matrix, which is used to generate multiple The boundary between audio objects; proportionally adjust the third similarity matrix to obtain the fourth similarity matrix.
  • the proportion adjustment is used to adjust the similarities included in each row of the third similarity matrix within the same range; adjust the fourth similarity matrix
  • the similarity matrix is symmetrized to obtain the similarity matrix corresponding to any reference parameter.
  • the determination module 501 is configured to adjust the similarity of any reference parameter other than the similarity that meets the third requirement to the third similarity for the multiple similarities included in each row of the reference similarity matrix. numerical value to obtain the first similarity matrix; or, among the multiple similarities included in the reference similarity matrix, the similarities other than the similarities that meet the third requirement of any reference parameter are multiplied by the fourth numerical value to obtain The first similarity matrix.
  • the determination module 501 is used to determine the transposed matrix corresponding to the first similarity matrix; The degrees are added to obtain a similarity matrix to be adjusted; a plurality of similarities included in the similarity matrix to be adjusted are halved to obtain a second similarity matrix.
  • the determination module 501 is used to determine the similarity located in the i-th row and j-th column of the first similarity matrix and the similarity located in the j-th row and i-th column of the first similarity matrix.
  • the maximum similarity is used as the similarity located in the i-th row, j-th column and j-th row, i-th column in the second similarity matrix to obtain the second similarity matrix.
  • the determination module 501 is used to determine the transposed matrix corresponding to the second similarity matrix; determine the third similarity based on the second similarity matrix and the transposed matrix corresponding to the second similarity matrix.
  • Matrix, the similarity in the m-th row and n-th column in the third similarity matrix is based on the similarity in the m-th row in the second similarity matrix and the similarity in the n-th column in the corresponding transposed matrix of the second similarity matrix Degree is determined, m and n are positive integers not greater than the number of multiple audio clips.
  • the determination module 501 is configured to determine the maximum similarity corresponding to each row according to the multiple similarities included in each row of the third similarity matrix; The similarity is divided by the maximum similarity corresponding to each row to obtain the fourth similarity matrix.
  • the determination module 501 is configured to determine the proportional value corresponding to each reference parameter according to the multiple reference parameters and the similarity matrix corresponding to each reference parameter.
  • the proportional value is used to indicate the similarity corresponding to the reference parameter.
  • the number of similarities retained in the matrix; the number of audio objects present in multiple audio clips is determined based on the proportional values corresponding to each reference parameter.
  • the determination module 501 is configured to perform a Laplace transform on the similarity matrix corresponding to any reference parameter among the plurality of reference parameters to obtain the corresponding Laplacian matrix; perform singular value decomposition on the Laplacian matrix to obtain multiple reference eigenvalues; determine the second eigenvalue and the first number of first eigenvalues and the second eigenvalue among the multiple reference eigenvalues The value is the maximum value among multiple reference eigenvalues, and the first eigenvalue is the reference eigenvalue that satisfies the second requirement after sorting the multiple reference eigenvalues in the second order; determine the relative value among the first number of first eigenvalues.
  • the difference between two adjacent first eigenvalues is used to obtain multiple eigenvalue differences; according to the second eigenvalue, the first eigenvalue difference is normalized to obtain the normalized eigenvalue difference. value, the first eigenvalue difference is the largest eigenvalue difference among multiple eigenvalue differences; according to the normalized eigenvalue difference and any reference parameter, the proportion value corresponding to any reference parameter is determined.
  • the determination module 501 is configured to determine a first parameter among multiple reference parameters according to the proportional value corresponding to each reference parameter.
  • the first parameter is the one with the smallest corresponding proportional value among the multiple reference parameters.
  • the clustering module 503 is used to perform singular value decomposition on the similarity matrix corresponding to the first parameter to obtain multiple decomposition eigenvalues; determine the relationship between the multiple decomposition eigenvalues and The number of audio objects corresponds to the decomposition eigenvalue; determine the eigenvectors corresponding to the number of decomposition eigenvalues of the audio object, and generate a decomposition matrix, the number of rows of the decomposition matrix is the number of the audio objects, and the number of columns is the number of the audio segments; according to the decomposition matrix, determine the feature vectors corresponding to the multiple audio segments, and the feature vectors are used to indicate the corresponding audio segments; according to the number of the audio objects and the multiple Feature vectors corresponding to each audio segment are clustered, and the audio segments obtained by each audio object's sound are clustered.
  • the above device adjusts the initial similarity matrix according to the dynamic threshold corresponding to each row in the initial similarity matrix, and then obtains the reference similarity matrix.
  • the similarity of the voiceprint vectors of the audio clips of the same audio object can be brought closer degree, zoom out the similarity of the voiceprint vectors of audio fragments of different audio objects, so that the number of audio objects determined based on the reference similarity matrix is more accurate; and then based on the number of audio objects with higher accuracy, multiple audio
  • the fragments are clustered to obtain the audio fragments corresponding to each audio object, so that the accuracy of the determined audio fragments corresponding to each audio object is higher, and the accuracy of audio object clustering is higher, which can improve the audio processing effect of the audio fragments.
  • FIG. 6 shows a structural block diagram of a terminal device 600 provided by an exemplary embodiment of the present application.
  • the terminal device 600 can be a portable mobile terminal, such as: a smartphone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Motion Picture Expert compresses standard audio levels 4) players, laptops or desktop computers.
  • the terminal device 600 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, and other names.
  • the terminal device 600 includes: a processor 601 and a memory 602.
  • the processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc.
  • the processor 601 can adopt at least one hardware form among DSP (Digital Signal Processing, digital signal processing), FPGA (Field-Programmable Gate Array, field programmable gate array), and PLA (Programmable Logic Array, programmable logic array).
  • the processor 601 may also include a main processor and a co-processor.
  • the main processor is a processor used to process data in the wake-up state, also called CPU (Central Processing Unit, central processing unit); the co-processor is A low-power processor used to process data in standby mode.
  • the processor 601 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is responsible for rendering and drawing content that needs to be displayed on the display screen.
  • the processor 601 may also include an AI (Artificial Intelligence, artificial intelligence) processor, which is used to process computing operations related to machine learning.
  • AI Artificial Intelligence, artificial intelligence
  • Memory 602 may include one or more computer-readable storage media, which may be non-transitory. Memory 602 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 601 to implement the audio processing method provided by the embodiment of the present application.
  • the terminal device 600 optionally further includes: a peripheral device interface 603 and at least one peripheral device.
  • the processor 601, the memory 602 and the peripheral device interface 603 may be connected through a bus or a signal line.
  • Each peripheral device can be connected to the peripheral device interface 603 through a bus, a signal line or a circuit board.
  • the peripheral device includes: at least one of a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio circuit 607 and a power supply 608.
  • the peripheral device interface 603 may be used to connect at least one I/O (Input/Output) related peripheral device to the processor 601 and the memory 602 .
  • the processor 601, the memory 602 and the peripheral device interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one of the processor 601, the memory 602 and the peripheral device interface 603 or Both can be implemented on separate chips or circuit boards, and this embodiment is not limited to this. Certainly.
  • the radio frequency circuit 604 is used to receive and transmit RF (Radio Frequency, radio frequency) signals, also called electromagnetic signals.
  • Radio frequency circuitry 604 communicates with communication networks and other communication devices through electromagnetic signals.
  • the radio frequency circuit 604 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals.
  • the radio frequency circuit 604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like.
  • the radio frequency circuit 604 can communicate with other terminal devices through at least one wireless communication protocol.
  • the wireless communication protocol includes but is not limited to: World Wide Web, metropolitan area network, intranet, mobile communication networks of all generations (2G, 3G, 4G and 5G), wireless LAN and/or WiFi (Wireless Fidelity, Wireless Fidelity) network.
  • the radio frequency circuit 604 may also include NFC (Near Field Communication) related circuits, which is not limited in this application.
  • the display screen 605 is used to display UI (User Interface, user interface).
  • the UI can include graphics, text, icons, videos, and any combination thereof.
  • display screen 605 is a touch display screen
  • display screen 605 also has the ability to collect touch signals on or above the surface of display screen 605 .
  • the touch signal can be input to the processor 601 as a control signal for processing.
  • the display screen 605 can also be used to provide virtual buttons and/or virtual keyboards, also called soft buttons and/or soft keyboards.
  • the display screen 605 may be a flexible display screen, disposed on the curved surface or folding surface of the terminal device 600. Even, the display screen 605 can also be set in a non-rectangular irregular shape, that is, a special-shaped screen.
  • the display screen 605 can be made of LCD (Liquid Crystal Display, liquid crystal display), OLED (Organic Light-Emitting Diode, organic light-emitting diode) and other materials.
  • the camera assembly 606 is used to capture images or videos.
  • the camera assembly 606 includes a front camera and a rear camera.
  • the front camera is arranged on the front panel of the terminal device 600 and the rear camera is arranged on the back of the terminal device 600 .
  • there are at least two rear cameras one of which is a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, so as to realize the integration of the main camera and the depth-of-field camera to realize the background blur function.
  • camera assembly 606 may also include a flash.
  • the flash can be a single color temperature flash or a dual color temperature flash. Dual color temperature flash refers to a combination of warm light flash and cold light flash, which can be used for light compensation under different color temperatures.
  • Audio circuitry 607 may include a microphone and speakers.
  • the microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 601 for processing, or to the radio frequency circuit 604 to implement voice communication. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal device 600 .
  • the microphone can also be an array microphone or an omnidirectional collection microphone.
  • the speaker is used to convert electrical signals from the processor 601 or the radio frequency circuit 604 into sound waves.
  • the loudspeaker can be a traditional membrane loudspeaker or a piezoelectric ceramic loudspeaker.
  • audio circuitry 607 may also include a headphone jack.
  • the power supply 608 is used to power various components in the terminal device 600 .
  • Power source 608 may be AC, DC, disposable batteries, or rechargeable batteries.
  • the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. Wired rechargeable batteries are batteries that are charged through wired lines, and wireless rechargeable batteries are batteries that are charged through wireless coils.
  • the rechargeable battery can also be used to support fast charging technology.
  • the terminal device 600 further includes one or more sensors 609.
  • the one or more sensors 609 include, but are not limited to: an acceleration sensor 610, a gyroscope sensor 611, a pressure sensor 612, an optical sensor 613, and a proximity sensor 614.
  • the acceleration sensor 610 can detect the acceleration on three coordinate axes of the coordinate system established by the terminal device 600 .
  • the acceleration sensor 610 can be used to detect the components of gravity acceleration on three coordinate axes.
  • the processor 601 can control the display screen 605 to operate in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 610. Display of the user interface.
  • the acceleration sensor 610 can also be used to collect game or user motion data.
  • the gyro sensor 611 can detect the body direction and rotation angle of the terminal device 600, and the gyro sensor 611 can cooperate with the acceleration sensor 610 to collect the user's 3D movements on the terminal device 600. Based on the data collected by the gyro sensor 611, the processor 601 can implement the following functions: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.
  • the pressure sensor 612 may be provided on the side frame of the terminal device 600 and/or on the lower layer of the display screen 605 .
  • the processor 601 performs left and right hand recognition or quick operation based on the grip signal collected by the pressure sensor 612.
  • the processor 601 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 605.
  • the operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.
  • the optical sensor 613 is used to collect ambient light intensity.
  • the processor 601 can control the display brightness of the display screen 605 according to the ambient light intensity collected by the optical sensor 613 . Specifically, when the ambient light intensity is high, the display brightness of the display screen 605 is increased; when the ambient light intensity is low, the display brightness of the display screen 605 is decreased.
  • the processor 601 can also dynamically adjust the shooting parameters of the camera assembly 606 according to the ambient light intensity collected by the optical sensor 613.
  • the proximity sensor 614 also called a distance sensor, is usually provided on the front panel of the terminal device 600.
  • the proximity sensor 614 is used to collect the distance between the user and the front of the terminal device 600 .
  • the processor 601 controls the display screen 605 to switch from the bright screen state to the closed screen state; when the proximity sensor 614 detects When the distance between the user and the front of the terminal device 600 gradually increases, the processor 601 controls the display screen 605 to switch from the screen-off state to the screen-on state.
  • FIG. 6 does not constitute a limitation on the terminal device 600, and may include more or fewer components than shown, or combine certain components, or adopt different component arrangements.
  • FIG. 7 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • the server 700 may vary greatly due to different configurations or performance, and may include one or more central processing units (Central Processing Units, CPUs) 701 and one or more A plurality of memories 702, wherein at least one program code is stored in the one or more memories 702, and the at least one program code is loaded and executed by the one or more processors 701 to implement the audio provided by the above method embodiments.
  • the server 700 may also have components such as wired or wireless network interfaces, keyboards, and input and output interfaces for input and output.
  • the server 700 may also include other components for implementing device functions, which will not be described again here.
  • a computer-readable storage medium is also provided, and at least one program code is stored in the storage medium, and the at least one program code is loaded and executed by the processor to enable the computer to implement any of the above audio Approach.
  • the above computer-readable storage medium may be read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), read-only compact disc (Compact Disc Read-Only Memory, CD-ROM) ), tapes, floppy disks and optical data storage devices, etc.
  • a computer program or computer program product is also provided. At least one computer instruction is stored in the computer program or computer program product, and the at least one computer instruction is loaded and executed by the processor, so that the computer implements Any of the above audio processing methods.
  • the information including but not limited to user equipment information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • signals involved in this application All are authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.
  • the audio clips involved in this application were obtained with full authorization.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Acoustics & Sound (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Game Theory and Decision Science (AREA)
  • Signal Processing (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种音频处理方法、装置、设备、可读存储介质及程序产品，属于计算机技术领域。方法包括：确定多个音频片段分别对应的声纹向量，声纹向量用于指示音频片段对应的声纹特征(201)；根据各个音频片段对应的声纹向量，确定初始相似度矩阵，初始相似度矩阵中包括任意两个音频片段对应的声纹向量之间的相似度(202)；根据初始相似度矩阵中各行对应的动态阈值，对初始相似度矩阵进行调整，得到参考相似度矩阵(203)；根据参考相似度矩阵确定多个音频片段中存在的音频对象的数目(204)；根据音频对象的数目，对多个音频片段进行聚类，得到各个音频对象对应的音频片段(205)。

Description

音频处理方法、装置、设备、可读存储介质及程序产品
本申请要求于2022年09月07日提交的申请号为202211088204.6、发明名称为“音频处理方法、装置、设备及计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及计算机技术领域，特别涉及一种音频处理方法、装置、设备、可读存储介质及程序产品。
背景技术
随着计算机技术的不断发展，音频处理方式也越来越多。例如，音频对象聚类是一种音频处理方式。音频对象聚类是确定多个音频片段中包括的音频对象的数目，以及各个音频对象对应的音频片段。
相关技术中，获取多个音频片段，确定各个音频片段对应的声纹向量，根据各个声纹向量之间的相似度，确定多个音频片段中存在的音频对象的数目，根据音频对象的数目对多个音频片段进行聚类，得到各个音频对象对应的音频片段。
然而，由于音频片段是通过录音设备获取到的，当录音设备、音频对象的语气、情绪不同时，会使得一个音频对象的音色产生偏差，且音频对象所处的环境也会对声纹向量的确定造成干扰，使得确定的声纹向量不够准确，导致音频对象聚类的准确率较低、音频处理效果较差。
发明内容
本申请实施例提供了一种音频处理方法、装置、设备、可读存储介质及程序产品，可以提高音频对象聚类的准确率。
一方面，本申请实施例提供了一种音频处理方法，由计算机设备执行，所述方法包括：
确定多个音频片段分别对应的声纹向量，所述声纹向量用于表示所述音频片段对应的声纹特征；
根据各个音频片段对应的声纹向量，确定初始相似度矩阵，所述初始相似度矩阵中包括任意两个音频片段对应的声纹向量之间的相似度；
根据所述初始相似度矩阵中各行对应的动态阈值，对所述初始相似度矩阵进行调整，得到参考相似度矩阵，所述动态阈值用于对不同相似度之间的相似度差值进行调节；
根据所述参考相似度矩阵确定发声得到所述多个音频片段的音频对象的数目；
根据所述音频对象的数目，对所述多个音频片段进行聚类，得到各个音频对象发声得到的音频片段。
另一方面，本申请实施例提供了一种音频处理装置，所述装置包括：
确定模块，用于确定多个音频片段分别对应的声纹向量，所述声纹向量用于表示所述音频片段对应的声纹特征；
所述确定模块，还用于根据各个音频片段对应的声纹向量，确定初始相似度矩阵，所述初始相似度矩阵中包括任意两个音频片段对应的声纹向量之间的相似度；
调整模块，用于根据所述初始相似度矩阵中各行对应的动态阈值，对所述初始相似度矩阵进行调整，得到参考相似度矩阵，所述动态阈值用于对不同相似度之间的相似度差值进行调节；
所述确定模块，还用于根据所述参考相似度矩阵确定发声得到所述多个音频片段的音频 对象的数目；
聚类模块，用于根据所述音频对象的数目，对所述多个音频片段进行聚类，得到各个音频对象发声得到的音频片段。
在一种可能的实现方式中，所述确定模块，还用于对于所述初始相似度矩阵中的任一行，按照第一顺序对所述任一行中位于预设相似度范围内的相似度进行排序，得到第一排序结果；确定所述第一排序结果中相邻的两个相似度之间的相似度差值，得到多个相似度差值；在所述多个相似度差值中确定满足第一要求的相似度差值；根据所述满足第一要求的相似度差值，确定所述任一行对应的动态阈值。
在一种可能的实现方式中，所述调整模块，用于将所述初始相似度矩阵第k行包括的相似度中，小于第k行对应的动态阈值的相似度调整为第一数值，并基于各行的调整结果得到所述参考相似度矩阵，k为正整数；或者，将所述初始相似度矩阵第k行包括的相似度中，小于所述第k行对应的动态阈值的相似度与第二数值相乘，并基于各行的调整结果得到所述参考相似度矩阵。
在一种可能的实现方式中，所述确定模块，用于根据多个参考参数，对所述参考相似度矩阵进行处理，得到各个参考参数对应的相似度矩阵；根据所述多个参考参数和所述各个参考参数对应的相似度矩阵，确定所述多个音频片段中存在的音频对象的数目。
在一种可能的实现方式中，所述确定模块，用于对于所述多个参考参数中的任一参考参数，根据所述任一参考参数，对所述参考相似度矩阵进行数值调整，得到第一相似度矩阵，所述数值调整用于简化所述参考相似度矩阵；对所述第一相似度矩阵进行对称化处理，得到第二相似度矩阵，所述第二相似度矩阵中位于第i行第j列的相似度与位于第j行第i列的相似度相同，所述i和所述j为不大于所述多个音频片段的个数的正整数；对所述第二相似度矩阵进行行列扩散，得到第三相似度矩阵，所述第三相似度矩阵用于生成多个音频对象之间的边界；对所述第三相似度矩阵进行比例调整，得到第四相似度矩阵，所述比例调整用于将所述第三相似度矩阵中各行包括的相似度调整在同一个范围内；对所述第四相似度矩阵进行对称化处理，得到所述任一参考参数对应的相似度矩阵。
在一种可能的实现方式中，所述确定模块，用于对于所述参考相似度矩阵各行包括的多个相似度，将任一参考参数个满足第三要求的相似度之外的相似度调整为第三数值，得到所述第一相似度矩阵；或者，将所述参考相似度矩阵包括的多个相似度中，除任一参考参数个满足第三要求的相似度之外的相似度与第四数值相乘，得到所述第一相似度矩阵。
在一种可能的实现方式中，所述确定模块，用于确定所述第一相似度矩阵对应的转置矩阵；将所述第一相似度矩阵和所述第一相似度矩阵对应的转置矩阵中位于相同位置的相似度相加，得到待调整相似度矩阵；对所述待调整相似度矩阵包括的多个相似度进行取半操作，得到所述第二相似度矩阵。
在一种可能的实现方式中，所述确定模块，用于确定所述第一相似度矩阵中位于所述第i行第j列的相似度，与所述第一相似度矩阵中位于所述第j行第i列的相似度中最大的相似度，将所述最大的相似度作为所述第二相似度矩阵中位于所述第i行第j列和所述第j行第i列的相似度，得到所述第二相似度矩阵。
在一种可能的实现方式中，所述确定模块，用于确定所述第二相似度矩阵对应的转置矩阵；根据所述第二相似度矩阵和所述第二相似度矩阵对应的转置矩阵，确定所述第三相似度矩阵，所述第三相似度矩阵中位于第m行第n列的相似度基于所述第二相似度矩阵中位于所述第m行的相似度和所述第二相似度矩阵对应的转置矩阵中位于所述第n列的相似度确定，所述m、所述n为不大于所述多个音频片段的个数的正整数。
在一种可能的实现方式中，所述确定模块，用于根据所述第三相似度矩阵中各行包括的多个相似度，确定各行对应的最大相似度；将所述第三相似度矩阵中各行包括的多个相似度分别与所述各行对应的最大相似度相除，得到所述第四相似度矩阵。
在一种可能的实现方式中，所述确定模块，用于根据所述多个参考参数和所述各个参考 参数对应的相似度矩阵，确定所述各个参考参数对应的比例值，所述比例值用于指示所述参考参数对应的相似度矩阵中保留的相似度的数量；根据所述各个参考参数对应的比例值，确定所述多个音频片段中存在的音频对象的数目。
在一种可能的实现方式中，所述确定模块，用于对于所述多个参考参数中的任一参考参数，对所述任一参考参数对应的相似度矩阵进行拉普拉斯变换，得到所述任一参考参数对应的拉普拉斯矩阵；对所述拉普拉斯矩阵进行奇异值分解，得到多个参考特征值；在所述多个参考特征值中确定第二特征值和第一数量个第一特征值，所述第二特征值为所述多个参考特征值中的最大值，所述第一特征值为按照第二顺序对所述多个参考特征值进行排序后满足第二要求的参考特征值；确定所述第一数量个第一特征值中相邻的两个第一特征值之间的差值，得到多个特征值差值；根据所述第二特征值，对第一特征值差值进行归一化处理，得到归一化之后的特征值差值，所述第一特征值差值为所述多个特征值差值中最大的特征值差值；根据所述归一化之后的特征值差值和所述任一参考参数，确定所述任一参考参数对应的比例值。
在一种可能的实现方式中，所述确定模块，用于根据所述各个参考参数对应的比例值，在所述多个参考参数中确定第一参数，所述第一参数为所述多个参考参数中对应的比例值最小的参考参数；确定所述第一参数对应的多个特征值差值；调用第一函数对所述第一参数对应的多个特征值差值进行处理，得到所述多个音频片段中存在的音频对象的数目。
在一种可能的实现方式中，所述聚类模块，用于对所述第一参数对应的相似度矩阵进行奇异值分解，得到多个分解特征值；在所述多个分解特征值中确定与所述音频对象的数目对应分解特征值；确定所述音频对象的数目个分解特征值分别对应的特征向量，并生成分解矩阵，所述分解矩阵的行数为所述音频对象的数目，列数为所述音频片段的数目；根据所述分解矩阵，确定所述多个音频片段分别对应的特征向量，所述特征向量用于指示对应的音频片段；根据所述音频对象的数目和所述多个音频片段分别对应的特征向量，对所述多个音频片段进行聚类，各个音频对象发声得到的音频片段。
另一方面，本申请实施例提供了一种计算机设备，所述计算机设备包括处理器和存储器，所述存储器中存储有至少一条程序代码，所述至少一条程序代码由所述处理器加载并执行，以使计算机设备实现上述任一所述的音频处理方法。
另一方面，还提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有至少一条程序代码，所述至少一条程序代码由处理器加载并执行，以使计算机实现上述任一所述的音频处理方法。
另一方面，还提供了一种计算机程序或计算机程序产品，所述计算机程序或计算机程序产品中存储有至少一条计算机指令，所述至少一条计算机指令由处理器加载并执行，以使计算机实现上述任一种音频处理方法。
本申请实施例提供的技术方案至少带来如下有益效果。
根据初始相似度矩阵中各行对应的动态阈值，对初始相似度矩阵进行调整，进而得到参考相似度矩阵，通过动态阈值调整过程，能够拉近同一音频对象的音频片段的声纹向量的相似度，拉远不同音频对象的音频片段的声纹向量的相似度，使得根据参考相似度矩阵，确定的音频对象的数目更加准确；再根据准确率较高的音频对象的数目，对多个音频片段进行聚类，得到各个音频对象对应的音频片段，使得确定的各个音频对象对应的音频片段的准确性较高，音频对象聚类的准确率较高，进而能够提高音频片段的音频处理效果。
附图说明
图1是本申请实施例提供的一种音频处理方法的实施环境示意图；
图2是本申请实施例提供的一种音频处理方法的流程图；
图3是本申请实施例提供的一种相似度矩阵的确定过程的示意图；
图4是本申请实施例提供的另一种音频处理方法的流程图；
图5是本申请实施例提供的一种音频处理装置的结构示意图；
图6是本申请实施例提供的一种终端设备的结构示意图；
图7是本申请实施例提供的一种服务器的结构示意图。
具体实施方式
在示例性实施例中，本申请实施例提供的音频处理方法可应用于各种场景，包括但不限于云技术、人工智能、智慧交通、辅助驾驶、游戏等。
人工智能(Artificial Intelligence，AI)是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用系统。换句话说，人工智能是计算机科学的一个综合技术，人工智能企图了解智能的实质，并生产出一种新的能以人类智能相似的方式做出反应的智能机器。人工智能也就是研究各种智能机器的设计原理与实现方法，使机器具有感知、推理与决策的功能。
本申请实施例提供的方案涉及人工智能技术中的机器学习技术，机器学习(Machine Learning，ML)是一门多领域交叉学科，涉及概率论、统计学、逼近论、凸分析、算法复杂度理论等多门学科。专门研究计算机怎样模拟或实现人类的学习行为，以获取新的知识或技能，重新组织已有的知识结构使之不断改善自身的性能。机器学习是人工智能的核心，是使计算机具有智能的根本途径，其应用遍及人工智能的各个领域。机器学习和深度学习通常包括人工神经网络、置信网络、强化学习、迁移学习、归纳学习、示教学习等技术。
随着人工智能技术研究和进步，人工智能技术在多个领域展开研究和应用，例如常见的智能家居、智能穿戴设备、虚拟助理、智能音箱、智能营销、无人驾驶、自动驾驶、无人机、机器人、智能医疗、智能客服、车联网、自动驾驶、智慧交通等。
图1是本申请实施例提供的一种音频处理方法的实施环境示意图，如图1所示，该实施环境包括：终端设备101和服务器102。
本申请实施例提供的音频处理方法可以由终端设备101执行，也可以由服务器102执行，还可以由终端设备101和服务器102共同执行，本申请实施例对此不进行限定。对于本申请实施例提供的音频处理方法由终端设备101和服务器102共同执行的情况，服务器102承担主要计算工作，终端设备101承担次要计算工作；或者，服务器102承担次要计算工作，终端设备101承担主要计算工作；或者，服务器102和终端设备101二者之间采用分布式计算架构进行协同计算。
可选地，终端设备101可以是任何一种可与用户通过键盘、触摸板、触摸屏、遥控器、语音交互或手写设备等一种或多种方式进行人机交互的电子产品。终端设备101包括但不限于手机、电脑、智能语音交互设备、智能家电、车载终端、飞行器等。服务器102为一台服务器，或者为多台服务器组成的服务器集群，或者为云计算平台和虚拟化中心，或者为区块链系统中的节点中的任意一种，本申请实施例对此不加以限定。服务器102与终端设备101通过有线网络或无线网络进行通信连接。服务器102具有数据接收功能、数据处理功能和数据发送功能。当然，服务器102还可以具有其他功能，本申请实施例对此不加以限定。
本领域技术人员应能理解上述终端设备101和服务器102仅为举例说明，其他现有的或者今后可能出现的终端设备或服务器，如可适用于本申请，也应包含在本申请的保护范围之内，并在此以引用方式包含于此。
本申请实施例提供了一种音频处理方法，该方法由计算机设备执行，该方法可应用于上述图1所示的实施环境，计算机设备可以是图1中的终端设备101，也可以是图1中的服务器102，本申请实施例对此不进行限定。以图2所示的本申请实施例提供的一种音频处理方法的流程图为例，如图2所示，该方法包括下述步骤201至步骤205。
在步骤201中，确定多个音频片段分别对应的声纹向量，声纹向量用于指示音频片段对应的声纹特征。
在本申请实施例中，在确定多个音频片段分别对应的声纹向量之前，需要先获取多个音频片段，多个音频片段为至少两个音频片段，每个音频片段对应于一个音频对象，音频对象为音频片段的发声对象，其中，不同音频片段对应的音频对象相同或者不同。本申请实施例对多个音频片段的获取过程不进行限定。示例性地，计算机设备的存储空间中存储有多个候选片段，从多个候选片段中获取多个音频片段。
可选地，还可以获取一个语音数据，语音数据中包括多个音频对象的音频片段，对语音数据进行分割处理，得到多个语音片段，在多个语音片段中确定待处理的多个音频片段。例如，将多个语音片段作为待处理的音频片段，或者在多个语音片段中选取一部分语音片段作为待处理的音频片段。示意性的，获取歌曲数据作为语音数据，根据歌词分段情况将语音数据分割为多个语音片段作为音频片段。
需要说明的是，多个音频片段分别对应的时长可以相同，也可以不同，本申请实施例对此不进行限定。例如，多个音频片段分别对应的时长均为2秒，又例如，多个音频片段中有的音频片段对应的时长为2秒，有的音频片段对应的时长为5秒。
可选地，确定出待处理的多个音频片段之后，对各个音频片段进行特征提取，得到各个音频片段对应的特征。根据各个音频片段对应的特征，确定各个音频片段对应的声纹向量。示例性地，各个音频片段对应的特征可以是各个音频片段对应的MFCC(Mel-scaleFrequency Cepstral Coefficients，梅尔倒谱系数)，也可以是各个音频片段对应的梅尔频谱特征，还可以是其他特征，本申请实施例对此不进行限定。
在一种可能的实现方式中，根据各个音频片段对应的特征，确定各个音频片段对应的声纹向量的过程包括：将音频片段对应的特征输入声纹提取模型，将声纹提取模型输出的结果作为音频片段对应的声纹向量。可选地，声纹提取模型可以是任意一种模型，本申请实施例对此不进行限定。示例性地，声纹提取模型可以为CLDNN(Convolution-Longshort-Term Mermony-Fully-Connected Deep Neural Networks，卷积-长短期记忆力-全连接神经网络)模型，也可以为基于TDNN(Time Delay Neural Network，时延神经网络)的X-vector(声纹识别领域主流的baseline模型框架)，还可以是ecapa-tdnn(一种提取语音全局特征的模型)。
在步骤202中，根据各个音频片段对应的声纹向量，确定初始相似度矩阵，初始相似度矩阵中包括任意两个音频片段对应的声纹向量之间的相似度。
在一种可能的实现方式中，在上述步骤201中确定出各个音频片段对应的声纹向量，根据各个音频片段对应的声纹向量，确定任意两个音频片段对应的声纹向量之间的相似度，得到初始相似度矩阵。
可选地，根据各个音频片段对应的声纹向量，按照下述公式(1)确定任意两个音频片段对应的声纹向量之间的相似度。
ai，j＝d(vi，vj) i，j∈[1，N]    公式(1)
在上述公式(1)中，ai，j为第i个音频片段对应的声纹向量和第j个音频片段对应的声纹向量之间的相似度，d(vi，vj)为距离公式，vi为第i个音频片段对应的声纹向量，vj为第j个音频片段对应的声纹向量，N为多个音频片段的总数量。
可选地，可以将任意两个音频片段对应的声纹向量之间的余弦相似度距离作为任意两个音频片段对应的声纹向量之间的相似度。当然，还可以通过其他方式确定任意两个音频片段对应的声纹向量之间的相似度，本申请实施例对此不进行限定。
需要说明的是，初始相似度矩阵的行数为多个音频片段的个数，列数为多个音频片段的个数。初始相似度矩阵为对称化矩阵，也即是初始相似度矩阵中位于第i行第j列的相似度与位于第j行第i列的相似度是相同的。其中，i，j均为不大于多个音频片段的个数的正整数。
还需要说明的是，任意两个音频片段对应的声纹向量之间的相似度越高，说明任意两个音频片段对应的音频对象是同一个音频对象的可能性较高。反之，任意两个音频片段对应的 声纹向量之间的相似度越低，说明任意两个音频片段对应的音频对象是同一个音频对象的可能性较低。
示例性地，多个音频片段的个数为5个，则根据各个音频片段对应的声纹向量，按照上述公式(1)确定任意两个音频片段的声纹向量之间的相似度。进而根据任意两个音频片段的声纹向量之间的相似度，确定的初始相似度矩阵为5*5的矩阵。初始相似度矩阵如下所示：
在上述初始相似度矩阵中，a1，1用于表示第1个音频片段对应的声纹向量和第1个音频片段对应的声纹向量之间的相似度，a1，2用于表示第1个音频片段对应的声纹向量和第2个音频片段对应的声纹向量之间的相似度，初始相似度矩阵中的其他元素所代表的含义与a1，1、a1，2所代表的含义类似，在此不再进行一一赘述。
在步骤203中，根据初始相似度矩阵中各行对应的动态阈值，对初始相似度矩阵进行调整，得到参考相似度矩阵。
其中，动态阈值用于对不同相似度之间的相似度差值进行调节。动态阈值用于拉近同一音频对象的音频片段的声纹向量的相似度之间的差值，和/或，拉远不同音频对象的音频片段的声纹向量的相似度之间的差值。可选地，拉近同一音频对象的音频片段的声纹向量的相似度之间的差值是指拉近第一相似度和第二相似度之间的差值，其中，第一相似度为第一音频片段的声纹向量和第二音频片段的声纹向量之间的相似度，第二相似度为第一音频片段的声纹向量和第三音频片段的声纹向量之间的相似度，第一音频片段、第二音频片段、第三音频片段对应于同一个音频对象。拉远不同音频对象的音频片段的声纹向量的相似度之间的差值是指拉远第一相似度和第三相似度之间的差值，第三相似度为第一音频片段的声纹向量和第四音频片段的声纹向量之间的相似度，第一音频片段、第四音频片段对应于不同的音频对象。
在一种可能的实现方式中，根据初始相似度矩阵中各行对应的动态阈值，对初始相似度矩阵进行调整，得到参考相似度矩阵之前，还需确定初始相似度矩阵中各行对应的动态阈值。该过程包括：对于初始相似度矩阵中的任一行，按照第一顺序对任一行包括的多个相似度中位于预设相似度范围的相似度进行排序，得到第一排序结果；根据第一排序结果，确定位于预设相似度范围内的相似度中相邻的两个相似度之间的相似度差值，得到多个相似度差值，相似度差值的个数小于任一行包括的相似度的个数，也即，确定第一排序结果中相邻的两个相似度之间的相似度差值，得到多个相似度差值；在多个相似度差值中确定满足第一要求的相似度差值；根据满足第一要求的相似度差值，确定任一行对应的动态阈值。可选地，将满足第一要求的相似度差值对应的被减数作为任一行对应的动态阈值。
其中，第一顺序可以是从小到大的顺序，也可以是从大到小的顺序，本申请实施例对此不进行限定。预设相似度范围是基于经验进行设置，或者是根据实施环境进行调整得到的，本申请实施例对此也不进行限定。示例性地，预设相似度范围为[-1，1]。多个相似度差值中满足第一要求的相似度差值是指多个相似度差值中最大的相似度差值。
示例性地，第一顺序为从小到大的顺序，预设相似度范围为[-1，1]，初始相似度矩阵中任一行包括的相似度分别为：1、-0.3、0.7、0.5、0.9。将位于预设相似度范围内的相似度按照从小到大的顺序进行排序，得到的第一排序结果为：-0.3、0.5、0.7、0.9、1。根据第一排序结果，确定相邻的两个相似度之间的相似度差值，得到多个相似度差值，分别为：0.5-(-0.3)＝0.8、0.7-0.5＝0.2、0.9-0.7＝0.2、1-0.9＝0.1。其中，多个相似度差值中最大的相似度差值为0.8，因此，将0.8对应的被减数0.5作为任一行对应的动态阈值。在一些实施例中，当存在多个与最大相似度差值对应的被减数时，确定任意一个被减数作为动态阈值。
可选地，根据第一排序结果，确定位于预设相似度范围内的相似度中相邻的两个相似度之间的相似度差值，得到多个相似度差值之后，还可以根据多个相似度差值确定相似度差值向量，相似度差值向量中包括多个相似度差值。示例性地，下述公式(2)为相似度差值向量。
gapq＝[a′q，2-a′q，1，a′q，3-a′q，2，...，a′q，N-a′q，N-1]    公式(2)
在上述公式(2)中，gapq任一行对应的相似度差值向量，a′q，1为按照从小到大的顺序排序后的第一排序结果中位于第一位的相似度，a′q，2为按照从小到大的顺序排序后的第一排序结果中位于第二位的相似度，a′q，3为按照从小到大的顺序排序后的第一排序结果中位于第三位的相似度，a′q，N为按照从小到大的顺序排序后的第一排序结果中位于最后一位的相似度，a′q，N-1为按照从小到大的顺序排序后的第一排序结果中位于倒数第二位的相似度。
在一种可能的实现方式中，确定出初始相似度矩阵中各行对应的动态阈值之后，根据初始相似度矩阵中各行对应的动态阈值，有下述两种实现方式对初始相似度矩阵进行调整，得到参考相似度矩阵。
实现方式一、将初始相似度矩阵第k行包括的相似度中，小于第k行对应的动态阈值的相似度调整为第一数值，并基于各行的调整结果得到参考相似度矩阵，k为正整数。
其中，第一数值基于经验进行设置，或者根据实施环境进行调整，本申请实施例对此不进行限定。可选地，第一数值为0。
示例性地，初始相似度矩阵为其中，第一行对应的动态阈值为0.5，第二行对应的动态阈值为0.6，第三行对应的动态阈值为0.7，第四行对应的动态阈值为0.2，第五行对应的动态阈值为0.9，则将初始相似度矩阵各行包括的相似度中，小于各行对应的动态阈值的相似度调整为0，得到的参考相似度矩阵为
实现方式二、将初始相似度矩阵第k行包括的相似度中，小于第k行对应的动态阈值的相似度与第二数值相乘，并基于各行的调整结果得到参考相似度矩阵。
其中，第二数值基于经验进行设置，或者根据实施环境进行调整，本申请实施例对此不进行限定。可选地，第二数值为0.01。
示例性地，初始相似度矩阵为其中，第一行对应的动态阈值为0.5，第二行对应的动态阈值为0.6，第三行对应的动态阈值为0.7，第四行对应的动态阈值为0.2，第五行对应的动态阈值为0.9，则将初始相似度矩阵各行包括的相似度中，小于各行对应的动态阈值的相似度与0.01相乘，得到的参考相似度矩阵为
需要说明的是，参考相似度矩阵中第一相似度和第二相似度之间的距离小于初始相似度矩阵中第一相似度和第二相似度之间的距离，参考相似度矩阵中第一相似度和第三相似度之间的距离大于初始相似度矩阵中第一相似度和第三相似度之间的距离，以达到拉近同一音频对象的音频片段的声纹向量的相似度之间的差值，拉远不同音频对象的音频片段的声纹向量之间的差值。
在步骤204中，根据参考相似度矩阵确定发声得到多个音频片段的音频对象的数目。
在一种可能的实现方式中，根据参考相似度矩阵确定多个音频片段中存在的音频对象的数目的过程包括：根据多个参考参数，对参考相似度矩阵进行处理，得到各个参考参数对应的相似度矩阵；根据多个参考参数和各个参考参数对应的相似度矩阵，确定多个音频片段中存在的音频对象的数目。其中，参考参数基于经验进行设置，或者根据实施环境进行调整，本申请实施例对此不进行限定。参考参数的个数本申请也不进行限定。
可选地，根据多个参考参数，对参考相似度矩阵进行处理，得到各个参考参数对应的相似度矩阵的过程是类似的，本申请实施例仅以多个参考参数中的任一个参考参数对应的相似度矩阵的确定过程为例进行说明，该过程包括下述步骤1至步骤5。
步骤1、根据任一参考参数，对参考相似度矩阵进行数值调整，得到第一相似度矩阵，数值调整用于简化参考相似度矩阵。
在一种可能的实现方式中，根据任一参考参数，有下述两种方式对参考相似度矩阵进行数值调整，得到第一相似度矩阵。
方式一、对于参考相似度矩阵各行包括的多个相似度，将任一参考参数个满足第三要求的相似度之外的相似度调整为第三数值，得到第一相似度矩阵。
其中，第三数值基于经验进行设置，或者根据实施环境进行调整，本申请实施例对此不进行限定。示例性地，第三数值为0。满足第三要求的任一参考参数个相似度是指任一参考参数个最大的相似度，也即，将与任意参考参数对应数量的相似度调整为第三数值，且被调整为第三数值的相似度是参考相似度矩阵行中最大的相似度。
可选地，将参考相似度矩阵各行包括的多个相似度分别按照从大到小的顺序进行排序，得到各行对应的排序结果，将各行对应的排序结果中，除了前任一参考参数个相似度之外的相似度调整为第三数值，得到第一相似度矩阵。
示例性地，任一参考参数为3，第三数值为0，参考相似度矩阵为根据任一参考参数，确定第一行中满足第三要求的3个相似度为1、0.9、0.7，第二行中满足第三要求的3个相似度为1、0.6、-0.003，第三行中满足第三要求的3个相似度为1、0.8、0.7，第四行中满足第三要求的3个相似度为1、0.8、0.5，第五行中满足第三要求的3个相似度为1、0.9、0.006。对参考相似度矩阵进行 调整，得到第一相似度矩阵为
方式二、将参考相似度矩阵包括的多个相似度中，除任一参考参数个满足第三要求的相似度之外的相似度与第四数值相乘，得到第一相似度矩阵。
其中，第四数值基于经验进行设置，或者根据实施环境进行调整，本申请实施例对此不进行限定。示例性地，第四数值为0.01。
可选地，将参考相似度矩阵各行包括的多个相似度分别按照从大到小的顺序进行排序，得到各行对应的排序结果，将各行对应的排序结果中除前任一参考参数个相似度之外的相似度与第四数值相乘，得到第一相似度矩阵，也即，将与任意参考参数对应数量的相似度以外的相似度与第四数值相乘，且与任意参考参数对应数量的相似度是参考相似度矩阵行中最大的相似度。
示例性地，任一参考参数为3，第四数值为0.01，参考相似度矩阵为根据任一参考参数，确定第一行中满足第三要求的3个相似度为1、0.9、0.7，第二行中满足第三要求的3个相似度为1、0.6、-0.003，第三行中满足第三要求的3个相似度为1、0.8、0.7，第四行中满足第三要求的3个相似度为1、0.8、0.5，第五行中满足第三要求的3个相似度为1、0.9、0.006。对参考相似度矩阵进行调整，得到第一相似度矩阵为参考相似度矩阵为
需要说明的是，可以选择上述任一种方式对参考相似度矩阵进行数值调整，得到第一相似度矩阵，本申请实施例对此不进行限定。
可选地，按照下述公式(3)对参考相似度矩阵进行数值调整，得到第一相似度矩阵。
B＝Threshold(A，p)     公式(3)
在上述公式(3)中，B为第一相似度矩阵，A为参考相似度矩阵，p为任一参考参数，Threshold为数值调整函数。
步骤2、对第一相似度矩阵进行对称化处理，得到第二相似度矩阵，所述第二相似度矩阵中位于第i行第j列的相似度与位于第j行第i列的相似度相同，i和j为不大于多个音频片段的个数的正整数。
由于初始相似度矩阵是对称化矩阵，经过动态阈值处理和数值调整后的第一相似度矩阵可能为非对称化矩阵，故对第一相似度矩阵进行对称化处理。由于第i个音频片段的声纹向量与第j个音频片段的声纹向量之间的相似度、第j个音频片段的声纹向量与第i个音频片段的声纹向量之间的相似度是相同的，也即是位于第i行第j列的相似度与位于第j行第i列的相似度是相同的，因此，需要对第一相似度矩阵进行对称化处理，以使位于第i行第j列的相似度与位于第j行第i列的相似度是相同的。可选地，至少存在下述方式对第一相似度矩阵进 行对称化处理，得到第二相似度矩阵。
方式1、确定第一相似度矩阵对应的转置矩阵；将第一相似度矩阵和第一相似度矩阵对应的转置矩阵中位于相同位置的相似度相加，得到待调整相似度矩阵；对待调整相似度矩阵包括的多个相似度进行取半操作，得到第二相似度矩阵。
可选地，按照下述公式(4)对第一相似度矩阵进行对称化处理，得到第二相似度矩阵。
在上述公式(4)中，C为第二相似度矩阵，B为第一相似度矩阵，BT为第一相似度矩阵对应的转置矩阵。
示例性地，第一相似度矩阵为第一相似度矩阵对应的转置矩阵为将第一相似度矩阵和第一相似度矩阵对应的转置矩阵中位于相同位置的相似度相加，得到的待调整相似度矩阵为对待调整相似度矩阵中包括的多个相似度进行取半操作，得到的第二相似度矩阵为
方式1为根据第一相似度矩阵和第一相似度矩阵对应的转置矩阵，通过取平均值的方式，来确定第二相似度矩阵的过程。
方式2、确定第一相似度矩阵中位于第i行第j列的相似度、与第一相似度矩阵中位于第j行第i列的相似度中最大的相似度，将最大的相似度作为第二相似度矩阵中位于第i行第j列和第j行第i列的相似度，得到第二相似度矩阵。
可选地，按照下述公式(5)对第一相似度矩阵进行对称化处理，得到第二相似度矩阵。
a′ij＝max(aij，aji)     公式(5)
在上述公式(5)中，a′i，j为第二相似度矩阵中位于第i行、第j列的相似度，aij为第一相似度矩阵中位于第i行、第j列的相似度，aji为第一相似度矩阵中位于第j行、第i列的相似度。
示例性地，第一相似度矩阵为则第二相似度矩阵为
方式2为根据第一相似度矩阵，通过取最大值的方式，确定第二相似度矩阵的过程。
需要说明的是，可以选择上述任一种方式对第一相似度矩阵进行对称化处理，得到第二相似度矩阵，本申请实施例对此不进行限定。
步骤3、对第二相似度矩阵进行行列扩散，得到第三相似度矩阵，第三相似度矩阵用于生成多个音频对象之间的边界。
在一种可能的实现方式中，对第二相似度矩阵进行行列扩散，得到第三相似度矩阵的过程包括：确定第二相似度矩阵对应的转置矩阵，根据第二相似度矩阵和第二相似度矩阵对应的转置矩阵，确定第三相似度矩阵，第三相似度矩阵中位于第m行第n列的相似度基于第二相似度矩阵中位于第m行的相似度和第二相似度矩阵对应的转置矩阵中位于第n列的相似度确定，m、n为不大于多个音频片段的个数的正整数。
可选地，对于第三相似度矩阵中位于第m行第n列的相似度，将第二相似度矩阵中位于第m行的相似度和第二相似度矩阵对应的转置矩阵中位于第n列的相似度对应相乘再相加的结果作为第三相似度矩阵中位于第m行第n列的相似度。
示例性地，第二相似度矩阵中位于第m行的相似度分别为1、0、0.7、0.5、0.9，第二相似度矩阵对应的转置矩阵中位于第n列的相似度分别为1、0、0.7、0.5、0.9，则第三相似度矩阵中位于第m行第n列的相似度为1*1+0*0+0.7*0.7+0.5*0.5+0.9*0.9＝2.55。
需要说明的是，第三相似度矩阵中其他位置的相似度的确定过程与上述第m行第n列的相似度的确定过程类似，在此不再进行赘述。
可选地，按照下述公式(6)对第二相似度矩阵进行行列扩散，得到第三相似度矩阵。
D＝CCT     公式(6)
在上述公式(6)中，D为第三相似度矩阵，C为第二相似度矩阵，CT为第二相似度矩阵对应的转置矩阵。
示例性地，第二相似度矩阵为第二相似度矩阵对应的转置矩阵为则第三相似度矩阵为
步骤4、对第三相似度矩阵进行比例调整，得到第四相似度矩阵，比例调整用于将第三相似度矩阵中各行包括的相似度调整在同一范围内。
在一种可能的实现方式中，对第三相似度矩阵进行比例调整，得到第四相似度矩阵的过程包括：根据第三相似度矩阵中各行包括的多个相似度，确定各行对应的最大相似度；将第三相似度矩阵中各行包括的多个相似度分别与各行对应的最大相似度相除，得到第四相似度矩阵。
可选地，按照下述公式(7)对第三相似度矩阵进行比例调整，得到第四相似度矩阵。
a″ij＝aij/aik     公式(7)
在上述公式(7)中，a″ij为第四相似度矩阵中位于第i行第j列的相似度，aij为第三相似度矩阵中位于第i行第j列的相似度，aik为第三相似度矩阵中第i行对应的最大相似度，k为第三相似度矩阵中第i行对应的最大相似度所在的列。
示例性地，第三相似度矩阵为其中，第一行对应的最大相似度为2.55，第二行对应的最大值为1.36，第三行对应的最大值为2.13，第四行对应的最大值为1.95，第五行对应的最大值为2.17。根据各行对应的最大相似度，对第三相似度矩阵进行比例调整，得到的第四相似度矩阵为
步骤5、对第四相似度矩阵进行对称化处理，得到任一参考参数对应的相似度矩阵。
在一种可能的实现方式中，对第四相似度矩阵进行对称化处理，得到任一参考参数对应的相似度矩阵的过程与上述对第一相似度矩阵进行对称化处理，得到第二相似度矩阵的过程是类似的，在此不再进行赘述。
需要说明的是，根据上述步骤1至步骤5的过程，分别确定出各个参考参数对应的相似度矩阵。
图3是本申请实施例提供的一种相似度矩阵的确定过程的示意图。图3中的(1)为初始相似度矩阵，图3中的(2)为参考相似度矩阵，图3中的(3)为第一相似度矩阵，图3中的(4)为第三相似度矩阵，图3中的(5)为第四相似度矩阵，图3中的(6)为参考参数对应的相似度矩阵。图3中的(1)中的横轴为音频片段的个数，纵轴为音频片段的个数，图3中区域亮度越高表示两个音频片段的声纹向量之间的相似度越高。
在一种可能的实现方式中，根据多个参考参数和各个参考参数对应的相似度矩阵，确定多个音频片段中存在的音频对象的数目的过程包括：根据多个参考参数和各个参考参数对应的相似度矩阵，确定各个参考参数对应的比例值，比例值用于指示参考参数对应的相似度矩阵中保留的相似度的数量；根据各个参考参数对应的比例值，确定多个音频片段中存在的音 频对象的数目。比例值越小，说明参考参数对应的相似度矩阵中保留的相似度的数量越少，后续确定的音频对象的数目的准确性越高；反之，比例值越大，说明参考参数对应的相似度矩阵中保留的相似度的数量越多，后续确定的音频对象的数目的准确性越低。
可选地，根据多个参考参数和各个参考参数对应的相似度矩阵，确定各个参考参数对应的比例值的过程包括：对于多个参考参数中的任一参考参数，对任一参考参数对应的相似度矩阵进行拉普拉斯变换，得到任一参考参数对应的拉普拉斯矩阵；对拉普拉斯矩阵进行奇异值分解，得到多个参考特征值；在多个参考特征值中确定第二特征值和第一数量个第一特征值，第二特征值为多个参考特征值中的最大值，第一特征值为按照第二顺序对多个参考特征值进行排序后满足第二要求的参考特征值。确定第一数量个第一特征值中相邻的两个第一特征值之间的差值，得到多个特征值差值；根据第二特征值，对第一特征值差值进行归一化处理，得到归一化之后的特征值差值，第一特征值差值为多个特征值差值中最大的特征值差值；根据归一化之后的特征值差值和任一参考参数，确定任一参考参数对应的比例值。其中，第一数量基于经验进行设置，或者根据实施环境进行调整，本申请实施例对此不进行限定。例如，第一数量为3。第二顺序可以是从小到大的顺序，也可以是从大到小的顺序，本申请实施例对此不进行限定。当第二顺序为从小到大的顺序时，则第一特征值为按照从小到大的顺序对多个参考特征值进行排序后，前第一数量个参考特征值。当第二顺序为从大到小的顺序时，则第一特征值为按照从大到小的顺序对多个参考特征值进行排序后，后第一数量个参考特征值。
可选地，确定第一数量个第一特征值中相邻的两个第一特征值之间的差值，得到多个特征值差值之后，还可以根据多个特征值差值确定任一参考参数对应的特征值差值向量，特征值差值向量中包括多个特征值差值。示例性地，下述公式(8)为任一参考参数对应的特征值差值向量。
ep＝[λp，2-λp，1，λp，3-λp，2，...，λp，Y-λp，Y-1]      公式(8)
在上述公式(8)中，ep为任一参考参数对应的特征值差值向量，λp，1为按照从小到大的顺序对多个参考特征值进行排序后位于第一位的参考特征值，λp，2为按照从小到大的顺序对多个参考特征值进行排序后位于第二位的参考特征值，λp，3为按照从小到大的顺序对多个参考特征值进行排序后位于第三位的参考特征值，λp，Y为按照从小到大的顺序对多个参考特征值进行排序后位于第Y位的参考特征值，λp，Y-1为按照从小到大的顺序对多个参考特征值进行排序后位于第Y-1位的相似度。Y为第一数量。
可选地，根据第二特征值，按照下述公式(9)对特征值差值进行归一化处理，得到归一化之后的特征值差值。
在上述公式(9)中，gP为归一化之后的特征值差值，max(ep)为第一特征值差值，λmax为第二特征值，ε为归一化参数，ε的取值为1*10-10。
根据归一化之后的特征值差值和任一参考参数，按照下述公式(10)确定任一参考参数对应的比例值。
在上述公式(10)中，r(p)为任一参考参数对应的比例值，p为任一参考参数，gp为归一化之后的特征值差值。
在一种可能的实现方式中，根据各个参考参数对应的比例值，确定多个音频片段中存在的音频对象的数目的过程包括：根据各个参考参数对应的比例值，在多个参考参数中确定第一参数，第一参数为多个参考参数中对应的比例值最小的参考参数；确定第一参数对应的多 个特征值差值，调用第一函数对第一参数对应的多个特征值差值进行处理，得到多个音频片段中存在的音频对象的数目。可选地，将各个参考参数对应的比例值中，最小的比例值对应的参考参数作为第一参数。
需要说明的是，确定第一参数对应的多个特征值差值的过程为：确定第一参数对应的相似度矩阵，对第一参数对应的相似度矩阵进行拉普拉斯变换，得到第一参数对应的拉普拉斯矩阵，对第一参数对应的拉普拉斯矩阵进行奇异值分解，得到多个参考特征值，将多个参考特征值中最小的第一数量个参考特征值进行排序，将排序中相邻的两个参考特征值之间的差值作为第一参数对应的多个特征值差值。
调用第一函数对第一参数对应的多个特征值差值进行处理，得到多个音频片段中存在的音频对象的数目的过程包括：将第一参数对应的多个特征值差值组成第一参数对应的特征值差值向量，调用第一函数对第一参数对应的特征值差值向量进行处理，得到多个音频片段中存在的音频对象的数目。
可选地，按照下述公式(11)对第一参数对应的多个特征值差值进行处理，得到多个音频片段中存在的音频对象的数目。
在上述公式(11)中，M为多个音频片段中存在的音频对象的数目，arg max()为第一函数，为第一参数对应的特征值差值向量，第一参数对应的特征值差值向量是由第一参数对应的多个特征值差值组成的向量。
示例性地，第一参数对应的相似度矩阵为矩阵Q，对矩阵Q进行拉普拉斯变换，得到第一参数对应的拉普拉斯矩阵P，对矩阵P进行奇异值分解，得到多个参考特征值(分别为a、b、c、d、e、f)，将多个参考特征值按照从小到大的顺序进行排序，得到排序结果(b、c、a、e、f、d)，第一数量为3，将排序结果中最小的3个参考值中相邻的两个参考特征值之间的差值作为第一参数对应的多个特征值差值，多个特征值差值分别为c-b、a-c，因此，将c-b、a-c组成的向量作为特征值差值向量，也即是特征值差值向量为[c-b，a-c]。
在步骤205中，根据音频对象的数目，对多个音频片段进行聚类，得到各个音频对象发声得到的音频片段。
在一种可能的实现方式中，基于上述步骤204确定出多个音频片段中存在的音频对象的数目之后，根据音频对象的数目，对多个音频片段进行聚类，得到各个音频对象对应的音频片段的过程包括：对第一参数对应的相似度矩阵进行奇异值分解，得到多个分解特征值；在多个分解特征值中确定音频对象的数目个分解特征值；确定音频对象的数目个分解特征值分别对应的特征向量；根据音频对象的数目个分解特征值分别对应的特征向量，生成分解矩阵，分解矩阵的行数为音频对象的数目，列数为音频片段的个数；根据分解矩阵，确定多个音频片段分别对应的特征向量，特征向量用于指示对应的音频片段；根据音频对象的数目和多个音频片段分别对应的特征向量，对多个音频片段进行聚类，得到各个音频对象对应的音频片段。
可选地，在多个分解特征值中确定音频对象的数目个分解特征值时，确定的分解特征值是最小的音频对象的数目个分解特征值。
示例性地，音频对象的数目为3，则在多个分解特征值中确定最小的3个分解特征值。确定这三个分解特征值分别对应的特征向量，分解特征值对应的特征向量为1*5的特征向量，将3个1*5的特征向量组成3*5的分解矩阵。将分解矩阵中第一列作为第一个音频片段对应的特征向量，第二列作为第二个音频片段对应的特征向量，第三列作为第三个音频片段对应的特征向量，第四列作为第四个音频片段对应的特征向量的，第五列作为第五个音频片段对应的特征向量。
示例性地，三个分解特征值分别对应的特征向量为[x1，x2，x3，x4，x5]、[y1，y2，y3，y4，y5]、[z1，z2，z3，z4，z5]，则根据三个分解特征值分别对应的特征向量，组成的分解矩阵为 因此，将[x1，y1，z1]作为第一个音频片段对应的特征向量，将[x2，y2，z2]作为第二个音频片段对应的特征向量，将[x3，y3，z3]作为第三个音频片段对应的特征向量，将[x4，y4，z4]作为第四个音频片段对应的特征向量，将[x5，y5，z5]作为第五个音频片段对应的特征向量。
可选地，根据音频对象的数目和多个音频片段分别对应的特征向量，通过K-means(K-均值)聚类算法对多个音频片段进行聚类，得到各个音频对象对应的音频片段，其中，K的取值为音频对象的数目。当然，还可以使用其他的聚类算法对多个音频片段进行聚类，本申请实施例对此不进行限定。
示例性地，待处理的音频片段有5个，分别为音频片段1、音频片段2、音频片段3、音频片段4和音频片段5。根据上述步骤201至步骤204，确定出待处理的音频片段中存在的音频对象的数目为3。根据上述步骤205，确定出音频对象1对应的音频片段为音频片段1、音频片段3，音频对象2对应的音频片段为音频片段5，音频对象3对应的音频片段为音频片段2和音频片段4。
在一种可能的实现方式中，在上述步骤204中确定出多个音频片段中存在的音频对象的数目之后，还可以根据音频对象的数目和多个音频片段分别对应的声纹向量，对多个音频片段进行聚类，得到各个音频对象对应的音频片段。
其中，根据音频对象的数目和多个音频片段分别对应的声纹向量，对多个音频片段进行聚类，得到各个音频对象对应的音频片段的过程与上述根据音频对象的数目和多个音频片段分别对应的特征向量，对多个音频片段进行聚类，得到各个音频对象对应的音频片段的过程类似，在此不再进行赘述。
本申请实施例提供的音频处理方法可应用于游戏领域中，确定同一个游戏账号(或同一个智能设备)由几个用户使用。可选地，采集用户在使用该游戏账号(或该智能设备)时的音频片段，调用本申请实施例提供的音频处理方法，确定各个音频片段对应的声纹向量，根据各个音频片段对应的声纹向量，确定音频片段中存在的音频对象的数目，以及各个音频对象对应的音频片段，以获知该游戏账号(或该智能设备)供几个用户使用。
上述方法根据初始相似度矩阵中各行对应的动态阈值，对初始相似度矩阵进行调整，进而得到参考相似度矩阵，通过动态阈值调整过程，能够拉近同一音频对象的音频片段的声纹向量的相似度，拉远不同音频对象的音频片段的声纹向量的相似度，使得根据参考相似度矩阵，确定的音频对象的数目更加准确；再根据准确率较高的音频对象的数目，对多个音频片段进行聚类，得到各个音频对象对应的音频片段，使得确定的各个音频对象对应的音频片段的准确性较高，音频对象聚类的准确率较高，进而能够提高音频片段的音频处理效果。
本实施例提供的方法，通过对初始相似度矩阵中的人一行进行相似度排序，从而根据相似度排序中各相邻相似度之间的差值确定动态阈值，从而以动态阈值对初始相似度矩阵进行调整，提高了拉近属于相同音频对象的声纹相似度以及拉远不同音频对象的声纹相似度的准确率，提高了确定音频对象数量的准确率。
本实施例提供的方法，基于动态阈值对初始相似度矩阵中的相似度进行第一数值的设置，提高了初始相似度矩阵的调整效率。
本实施例提供的方法，基于动态阈值对初始相似度矩阵中的相似度与第二数值进行运算设置，提高了调整后的参考相似度矩阵的设置灵活性和准确率。
本实施例提供的方法，生成多个参考参数，并基于多个参考参数对参考相似度矩阵进行处理，得到各个参考参数对应的相似度矩阵，从而提高了相似度矩阵的准确率，并提高了确定音频对象数目的准确率。
本实施例提供的方法，在确定参考参数对应的相似度矩阵时，对参考相似度矩阵进行调 整后，由于调整后的第一相似度矩阵存在不对称的可能，故对第一相似度矩阵进行对称化处理，并再进行行列扩散和比例调整处理，提高了确定相似度矩阵的准确率。
本实施例提供的方法，在确定第一相似度矩阵时，根据第三要求确定任一参考参数个相似度，并将其他相似度进行数值调整，并通过第一相似度矩阵和对应的转置矩阵确定第二相似度矩阵，提高了参考参数对应的相似度矩阵的准确率。
图4是本申请实施例提供的另一种音频处理方法的流程图，如图4所示，该方法包括下述步骤401至步骤415。
401、获取多个音频片段。
在一种可能的实现方式中，该过程已在上述步骤201中进行描述，在此不再进行赘述。
402、对各个音频片段进行信号预处理，得到各个音频片段的特征。
在一种可能的实现方式中，该过程已在上述步骤201中进行描述，在此不再进行赘述。其中，对音频片段进行信号预处理的方式包括分割、降噪、采样、量化等处理方式中的至少一种。
403、调用声纹提取模型对各个音频片段的特征进行处理，得到各个音频片段对应的声纹向量。
在一种可能的实现方式中，该过程已在上述步骤201中进行描述，在此不再进行赘述。示例性地，声纹提取模型可以为CLDNN模型，也可以为基于TDNN的X-vector，还可以是ecapa-tdnn。
404、根据各个音频片段对应的声纹向量，确定初始相似度矩阵。
在一种可能的实现方式中，该过程已在上述步骤202中进行描述，在此不再进行赘述。
其中，确定任意两个音频片段对应的声纹向量之间的相似度，并构建初始相似度矩阵，其中，初始相似度矩阵的纵向和横向的数量与音频片段的数量对应，从而构建各个音频片段的声纹向量之间的相似度的对应矩阵作为初始相似度矩阵。
405、根据初始相似度矩阵中各行对应的动态阈值，对初始相似度矩阵进行调整，得到参考相似度矩阵。
在一种可能的实现方式中，该过程已在上述步骤203中进行描述，在此不再进行赘述。其中，动态阈值用于拉近相同发声对象对应的音频片段的声纹特征之间的相似度，或者，动态阈值用于拉远不同发声对象对应的音频片段的声纹特征之间的相似度。
406、根据多个参考参数，对参考相似度矩阵进行数值调整，得到各个参考参数对应的第一相似度矩阵。
在一种可能的实现方式中，该过程已在上述步骤204中进行描述，在此不再进行赘述。可选地，对于参考相似度矩阵各行包括的多个相似度，将任一参考参数个满足第三要求的相似度之外的相似度调整为第三数值，得到第一相似度矩阵；或者，将参考相似度矩阵包括的多个相似度中，除任一参考参数个满足第三要求的相似度之外的相似度与第四数值相乘，得到第一相似度矩阵。
407、对各个参考参数对应的第一相似度矩阵进行对称化处理，得到各个参考参数对应的第二相似度矩阵。
在一种可能的实现方式中，该过程已在上述步骤204中进行描述，在此不再进行赘述。由于参考相似度矩阵被调整为第一相似度矩阵时，第一相似度矩阵存在不对称情况，故对第一相似度矩阵进行对称化处理，得到第二相似度矩阵。
408、对各个参考参数对应的第二相似度矩阵进行行列扩散，得到各个参考参数对应的第三相似度矩阵。
在一种可能的实现方式中，该过程已在上述步骤204中进行描述，在此不再进行赘述。其中，第二相似度矩阵通过转置矩阵进行行列扩散，得到第三转置矩阵。
409、对各个参考参数对应的第三相似度矩阵进行比例调整，得到各个参考参数对应的第 四相似度矩阵。
在一种可能的实现方式中，该过程已在上述步骤204中进行描述，在此不再进行赘述。可选地，根据第三相似度矩阵中各行包括的多个相似度，确定各行对应的最大相似度；将第三相似度矩阵中各行包括的多个相似度分别与各行对应的最大相似度相除，得到第四相似度矩阵。
410、对各个参考参数对应的第四相似度矩阵进行对称化处理，得到各个参考参数对应的相似度矩阵。
在一种可能的实现方式中，该过程已在上述步骤204中进行描述，在此不再进行赘述。由于第三相似度矩阵进行比例调整后，第四相似度矩阵存在不对称情况，故对第四相似度矩阵进行对称化处理，得到参考参数对应的相似度矩阵。
411、根据多个参考参数和各个参考参数对应的相似度矩阵，确定各个参考参数对应的比例值。
在一种可能的实现方式中，该过程已在上述步骤204中进行描述，在此不再进行赘述。比例值用于指示参考参数对应的相似度矩阵中保留的相似度的数量。比例值越小，说明参考参数对应的相似度矩阵中保留的相似度的数量越少，后续确定的音频对象的数目的准确性越高；反之，比例值越大，说明参考参数对应的相似度矩阵中保留的相似度的数量越多，后续确定的音频对象的数目的准确性越低。
412、根据各个参考参数对应的比例值，在多个参考参数中确定第一参数。
在一种可能的实现方式中，该过程已在上述步骤204中进行描述，在此不再进行赘述。
413、根据第一参数，确定多个音频片段中存在的音频对象的数目。
在一种可能的实现方式中，该过程已在上述步骤204中进行描述，在此不再进行赘述。
414、根据第一参数，确定各个音频片段对应的特征向量。
在一种可能的实现方式中，该过程已在上述步骤205中进行描述，在此不再进行赘述。
415、根据音频对象的数目和各个音频片段对应的特征向量，对多个音频片段进行聚类，得到各个音频对象对应的音频片段。
在一种可能的实现方式中，该过程已在上述步骤205中进行描述，在此不再进行赘述。
图5所示为本申请实施例提供的一种音频处理装置的结构示意图，如图5所示，该装置包括：
确定模块501，用于确定多个音频片段分别对应的声纹向量，声纹向量用于指示音频片段对应的声纹特征；
确定模块501，还用于根据各个音频片段对应的声纹向量，确定初始相似度矩阵，初始相似度矩阵中包括任意两个音频片段对应的声纹向量之间的相似度；
调整模块502，用于根据初始相似度矩阵中各行对应的动态阈值，对初始相似度矩阵进行调整，得到参考相似度矩阵，所述动态阈值用于对不同相似度之间的相似度差值进行调节；
确定模块501，还用于根据所述参考相似度矩阵确定发声得到所述多个音频片段的音频对象的数目；
聚类模块503，用于根据所述音频对象的数目，对所述多个音频片段进行聚类，得到各个音频对象发声得到的音频片段。
在一种可能的实现方式中，确定模块501，还用于对于所述初始相似度矩阵中的任一行，按照第一顺序对所述任一行中位于预设相似度范围内的相似度进行排序，得到第一排序结果；确定所述第一排序结果中相邻的两个相似度之间的相似度差值，得到多个相似度差值；在多个相似度差值中确定满足第一要求的相似度差值；根据满足第一要求的相似度差值，确定任一行对应的动态阈值。
在一种可能的实现方式中，调整模块502，用于将所述初始相似度矩阵第k行包括的相似度中，小于第k行对应的动态阈值的相似度调整为第一数值，并基于各行的调整结果得到 所述参考相似度矩阵，k为正整数；或者，将所述初始相似度矩阵第k行包括的相似度中，小于所述第k行对应的动态阈值的相似度与第二数值相乘，并基于各行的调整结果得到所述参考相似度矩阵。
在一种可能的实现方式中，确定模块501，用于根据多个参考参数，对参考相似度矩阵进行处理，得到各个参考参数对应的相似度矩阵；根据多个参考参数和各个参考参数对应的相似度矩阵，确定多个音频片段中存在的音频对象的数目。
在一种可能的实现方式中，确定模块501，用于对于多个参考参数中的任一参考参数，根据任一参考参数，对参考相似度矩阵进行数值调整，得到第一相似度矩阵，数值调整用于简化参考相似度矩阵；对第一相似度矩阵进行对称化处理，得到第二相似度矩阵，第二相似度矩阵中位于第i行第j列的相似度与位于第j行第i列的相似度相同，i和j为不大于多个音频片段的个数的正整数；对第二相似度矩阵进行行列扩散，得到第三相似度矩阵，第三相似度矩阵用于生成多个音频对象之间的边界；对第三相似度矩阵进行比例调整，得到第四相似度矩阵，比例调整用于将第三相似度矩阵中各行包括的相似度调整在同一个范围内；对第四相似度矩阵进行对称化处理，得到任一参考参数对应的相似度矩阵。
在一种可能的实现方式中，确定模块501，用于对于参考相似度矩阵各行包括的多个相似度，将任一参考参数个满足第三要求的相似度之外的相似度调整为第三数值，得到第一相似度矩阵；或者，将参考相似度矩阵包括的多个相似度中，除任一参考参数个满足第三要求的相似度之外的相似度与第四数值相乘，得到第一相似度矩阵。
在一种可能的实现方式中，确定模块501，用于确定第一相似度矩阵对应的转置矩阵；将第一相似度矩阵和第一相似度矩阵对应的转置矩阵中位于相同位置的相似度相加，得到待调整相似度矩阵；对待调整相似度矩阵包括的多个相似度进行取半操作，得到第二相似度矩阵。
在一种可能的实现方式中，确定模块501，用于确定第一相似度矩阵中位于第i行第j列的相似度，与第一相似度矩阵中位于第j行第i列的相似度中最大的相似度，将最大的相似度作为第二相似度矩阵中位于第i行第j列和第j行第i列的相似度，得到第二相似度矩阵。
在一种可能的实现方式中，确定模块501，用于确定第二相似度矩阵对应的转置矩阵；根据第二相似度矩阵和第二相似度矩阵对应的转置矩阵，确定第三相似度矩阵，第三相似度矩阵中位于第m行第n列的相似度基于第二相似度矩阵中位于第m行的相似度和第二相似度矩阵对应的转置矩阵中位于第n列的相似度确定，m、n为不大于多个音频片段的个数的正整数。
在一种可能的实现方式中，确定模块501，用于根据第三相似度矩阵中各行包括的多个相似度，确定各行对应的最大相似度；将第三相似度矩阵中各行包括的多个相似度分别与各行对应的最大相似度相除，得到第四相似度矩阵。
在一种可能的实现方式中，确定模块501，用于根据多个参考参数和各个参考参数对应的相似度矩阵，确定各个参考参数对应的比例值，比例值用于指示参考参数对应的相似度矩阵中保留的相似度的数量；根据各个参考参数对应的比例值，确定多个音频片段中存在的音频对象的数目。
在一种可能的实现方式中，确定模块501，用于对于多个参考参数中的任一参考参数，对任一参考参数对应的相似度矩阵进行拉普拉斯变换，得到任一参考参数对应的拉普拉斯矩阵；对拉普拉斯矩阵进行奇异值分解，得到多个参考特征值；在多个参考特征值中确定第二特征值和第一数量个第一特征值，第二特征值为多个参考特征值中的最大值，第一特征值为按照第二顺序对多个参考特征值进行排序后满足第二要求的参考特征值；确定第一数量个第一特征值中相邻的两个第一特征值之间的差值，得到多个特征值差值；根据第二特征值，对第一特征值差值进行归一化处理，得到归一化之后的特征值差值，第一特征值差值为多个特征值差值中最大的特征值差值；根据归一化之后的特征值差值和任一参考参数，确定任一参考参数对应的比例值。
在一种可能的实现方式中，确定模块501，用于根据各个参考参数对应的比例值，在多个参考参数中确定第一参数，第一参数为多个参考参数中对应的比例值最小的参考参数；确定第一参数对应的多个特征值差值；调用第一函数对第一参数对应的多个特征值差值进行处理，得到多个音频片段中存在的音频对象的数目。
在一种可能的实现方式中，聚类模块503，用于对所述第一参数对应的相似度矩阵进行奇异值分解，得到多个分解特征值；在所述多个分解特征值中确定与所述音频对象的数目对应分解特征值；确定所述音频对象的数目个分解特征值分别对应的特征向量，并生成分解矩阵，所述分解矩阵的行数为所述音频对象的数目，列数为所述音频片段的数目；根据所述分解矩阵，确定所述多个音频片段分别对应的特征向量，所述特征向量用于指示对应的音频片段；根据所述音频对象的数目和所述多个音频片段分别对应的特征向量，对所述多个音频片段进行聚类，各个音频对象发声得到的音频片段。
上述装置根据初始相似度矩阵中各行对应的动态阈值，对初始相似度矩阵进行调整，进而得到参考相似度矩阵，通过动态阈值调整过程，能够拉近同一音频对象的音频片段的声纹向量的相似度，拉远不同音频对象的音频片段的声纹向量的相似度，使得根据参考相似度矩阵，确定的音频对象的数目更加准确；再根据准确率较高的音频对象的数目，对多个音频片段进行聚类，得到各个音频对象对应的音频片段，使得确定的各个音频对象对应的音频片段的准确性较高，音频对象聚类的准确率较高，进而能够提高音频片段的音频处理效果。
图6示出了本申请一个示例性实施例提供的终端设备600的结构框图。该终端设备600可以是便携式移动终端，比如：智能手机、平板电脑、MP3播放器(Moving Picture Experts Group Audio Layer III，动态影像专家压缩标准音频层面3)、MP4(Moving Picture Experts Group Audio Layer IV，动态影像专家压缩标准音频层面4)播放器、笔记本电脑或台式电脑。终端设备600还可能被称为用户设备、便携式终端、膝上型终端、台式终端等其他名称。
通常，终端设备600包括有：处理器601和存储器602。
处理器601可以包括一个或多个处理核心，比如4核心处理器、8核心处理器等。处理器601可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field-Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。处理器601也可以包括主处理器和协处理器，主处理器是用于对在唤醒状态下的数据进行处理的处理器，也称CPU(Central Processing Unit，中央处理器)；协处理器是用于对在待机状态下的数据进行处理的低功耗处理器。在一些实施例中，处理器601可以集成有GPU(Graphics Processing Unit，图像处理器)，GPU用于负责显示屏所需要显示的内容的渲染和绘制。一些实施例中，处理器601还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。
存储器602可以包括一个或多个计算机可读存储介质，该计算机可读存储介质可以是非暂态的。存储器602还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。在一些实施例中，存储器602中的非暂态的计算机可读存储介质用于存储至少一个指令，该至少一个指令用于被处理器601所执行以实现本申请实施例提供的音频处理方法。
在一些实施例中，终端设备600还可选包括有：外围设备接口603和至少一个外围设备。处理器601、存储器602和外围设备接口603之间可以通过总线或信号线相连。各个外围设备可以通过总线、信号线或电路板与外围设备接口603相连。具体地，外围设备包括：射频电路604、显示屏605、摄像头组件606、音频电路607和电源608中的至少一种。
外围设备接口603可被用于将I/O(Input/Output，输入/输出)相关的至少一个外围设备连接到处理器601和存储器602。在一些实施例中，处理器601、存储器602和外围设备接口603被集成在同一芯片或电路板上；在一些其他实施例中，处理器601、存储器602和外围设备接口603中的任意一个或两个可以在单独的芯片或电路板上实现，本实施例对此不加以限 定。
射频电路604用于接收和发射RF(Radio Frequency，射频)信号，也称电磁信号。射频电路604通过电磁信号与通信网络以及其他通信设备进行通信。射频电路604将电信号转换为电磁信号进行发送，或者，将接收到的电磁信号转换为电信号。可选地，射频电路604包括：天线系统、RF收发器、一个或多个放大器、调谐器、振荡器、数字信号处理器、编解码芯片组、用户身份模块卡等等。射频电路604可以通过至少一种无线通信协议来与其它终端设备进行通信。该无线通信协议包括但不限于：万维网、城域网、内联网、各代移动通信网络(2G、3G、4G及5G)、无线局域网和/或WiFi(Wireless Fidelity，无线保真)网络。在一些实施例中，射频电路604还可以包括NFC(Near Field Communication，近距离无线通信)有关的电路，本申请对此不加以限定。
显示屏605用于显示UI(User Interface，用户界面)。该UI可以包括图形、文本、图标、视频及其它们的任意组合。当显示屏605是触摸显示屏时，显示屏605还具有采集在显示屏605的表面或表面上方的触摸信号的能力。该触摸信号可以作为控制信号输入至处理器601进行处理。此时，显示屏605还可以用于提供虚拟按钮和/或虚拟键盘，也称软按钮和/或软键盘。在一些实施例中，显示屏605可以为一个，设置在终端设备600的前面板；在另一些实施例中，显示屏605可以为至少两个，分别设置在终端设备600的不同表面或呈折叠设计；在另一些实施例中，显示屏605可以是柔性显示屏，设置在终端设备600的弯曲表面上或折叠面上。甚至，显示屏605还可以设置成非矩形的不规则图形，也即异形屏。显示屏605可以采用LCD(Liquid Crystal Display，液晶显示屏)、OLED(Organic Light-Emitting Diode，有机发光二极管)等材质制备。
摄像头组件606用于采集图像或视频。可选地，摄像头组件606包括前置摄像头和后置摄像头。通常，前置摄像头设置在终端设备600的前面板，后置摄像头设置在终端设备600的背面。在一些实施例中，后置摄像头为至少两个，分别为主摄像头、景深摄像头、广角摄像头、长焦摄像头中的任意一种，以实现主摄像头和景深摄像头融合实现背景虚化功能、主摄像头和广角摄像头融合实现全景拍摄以及VR(Virtual Reality，虚拟现实)拍摄功能或者其它融合拍摄功能。在一些实施例中，摄像头组件606还可以包括闪光灯。闪光灯可以是单色温闪光灯，也可以是双色温闪光灯。双色温闪光灯是指暖光闪光灯和冷光闪光灯的组合，可以用于不同色温下的光线补偿。
音频电路607可以包括麦克风和扬声器。麦克风用于采集用户及环境的声波，并将声波转换为电信号输入至处理器601进行处理，或者输入至射频电路604以实现语音通信。出于立体声采集或降噪的目的，麦克风可以为多个，分别设置在终端设备600的不同部位。麦克风还可以是阵列麦克风或全向采集型麦克风。扬声器则用于将来自处理器601或射频电路604的电信号转换为声波。扬声器可以是传统的薄膜扬声器，也可以是压电陶瓷扬声器。当扬声器是压电陶瓷扬声器时，不仅可以将电信号转换为人类可听见的声波，也可以将电信号转换为人类听不见的声波以进行测距等用途。在一些实施例中，音频电路607还可以包括耳机插孔。
电源608用于为终端设备600中的各个组件进行供电。电源608可以是交流电、直流电、一次性电池或可充电电池。当电源608包括可充电电池时，该可充电电池可以是有线充电电池或无线充电电池。有线充电电池是通过有线线路充电的电池，无线充电电池是通过无线线圈充电的电池。该可充电电池还可以用于支持快充技术。
在一些实施例中，终端设备600还包括有一个或多个传感器609。该一个或多个传感器609包括但不限于：加速度传感器610、陀螺仪传感器611、压力传感器612、光学传感器613以及接近传感器614。
加速度传感器610可以检测以终端设备600建立的坐标系的三个坐标轴上的加速度大小。比如，加速度传感器610可以用于检测重力加速度在三个坐标轴上的分量。处理器601可以根据加速度传感器610采集的重力加速度信号，控制显示屏605以横向视图或纵向视图进行 用户界面的显示。加速度传感器610还可以用于游戏或者用户的运动数据的采集。
陀螺仪传感器611可以检测终端设备600的机体方向及转动角度，陀螺仪传感器611可以与加速度传感器610协同采集用户对终端设备600的3D动作。处理器601根据陀螺仪传感器611采集的数据，可以实现如下功能：动作感应(比如根据用户的倾斜操作来改变UI)、拍摄时的图像稳定、游戏控制以及惯性导航。
压力传感器612可以设置在终端设备600的侧边框和/或显示屏605的下层。当压力传感器612设置在终端设备600的侧边框时，可以检测用户对终端设备600的握持信号，由处理器601根据压力传感器612采集的握持信号进行左右手识别或快捷操作。当压力传感器612设置在显示屏605的下层时，由处理器601根据用户对显示屏605的压力操作，实现对UI界面上的可操作性控件进行控制。可操作性控件包括按钮控件、滚动条控件、图标控件、菜单控件中的至少一种。
光学传感器613用于采集环境光强度。在一个实施例中，处理器601可以根据光学传感器613采集的环境光强度，控制显示屏605的显示亮度。具体地，当环境光强度较高时，调高显示屏605的显示亮度；当环境光强度较低时，调低显示屏605的显示亮度。在另一个实施例中，处理器601还可以根据光学传感器613采集的环境光强度，动态调整摄像头组件606的拍摄参数。
接近传感器614，也称距离传感器，通常设置在终端设备600的前面板。接近传感器614用于采集用户与终端设备600的正面之间的距离。在一个实施例中，当接近传感器614检测到用户与终端设备600的正面之间的距离逐渐变小时，由处理器601控制显示屏605从亮屏状态切换为息屏状态；当接近传感器614检测到用户与终端设备600的正面之间的距离逐渐变大时，由处理器601控制显示屏605从息屏状态切换为亮屏状态。
本领域技术人员可以理解，图6中示出的结构并不构成对终端设备600的限定，可以包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。
图7为本申请实施例提供的服务器的结构示意图，该服务器700可因配置或性能不同而产生比较大的差异，可以包括一个或多个中央处理器(Central Processing Units，CPU)701和一个或多个的存储器702，其中，该一个或多个存储器702中存储有至少一条程序代码，该至少一条程序代码由该一个或多个处理器701加载并执行以实现上述各个方法实施例提供的音频处理方法。当然，该服务器700还可以具有有线或无线网络接口、键盘以及输入输出接口等部件，以便进行输入输出，该服务器700还可以包括其他用于实现设备功能的部件，在此不做赘述。
在示例性实施例中，还提供了一种计算机可读存储介质，该存储介质中存储有至少一条程序代码，该至少一条程序代码由处理器加载并执行，以使计算机实现上述任一种音频处理方法。
可选地，上述计算机可读存储介质可以是只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、只读光盘(Compact Disc Read-Only Memory，CD-ROM)、磁带、软盘和光数据存储设备等。
在示例性实施例中，还提供了一种计算机程序或计算机程序产品，该计算机程序或计算机程序产品中存储有至少一条计算机指令，该至少一条计算机指令由处理器加载并执行，以使计算机实现上述任一种音频处理方法。
需要说明的是，本申请所涉及的信息(包括但不限于用户设备信息、用户个人信息等)、数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)以及信号，均为经用户授权或者经过各方充分授权的，且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。例如，本申请中涉及到的音频片段都是在充分授权的情况下获取的。

Claims (20)

  1. 一种音频处理方法，由计算机设备执行，所述方法包括：
    确定多个音频片段分别对应的声纹向量，所述声纹向量用于表示所述音频片段对应的声纹特征；
    根据各个音频片段对应的声纹向量，确定初始相似度矩阵，所述初始相似度矩阵中包括任意两个音频片段对应的声纹向量之间的相似度；
    根据所述初始相似度矩阵中各行对应的动态阈值，对所述初始相似度矩阵进行调整，得到参考相似度矩阵，所述动态阈值用于对不同相似度之间的相似度差值进行调节；
    根据所述参考相似度矩阵确定发声得到所述多个音频片段的音频对象的数目；
    根据所述音频对象的数目，对所述多个音频片段进行聚类，得到各个音频对象发声得到的音频片段。
  2. 根据权利要求1所述的方法，其中，所述根据所述初始相似度矩阵中各行对应的动态阈值，对所述初始相似度矩阵进行调整，得到参考相似度矩阵之前，所述方法还包括：
    对于所述初始相似度矩阵中的任一行，按照第一顺序对所述任一行中位于预设相似度范围内的相似度进行排序，得到第一排序结果；
    确定所述第一排序结果中相邻的两个相似度之间的相似度差值，得到多个相似度差值；
    在所述多个相似度差值中确定满足第一要求的相似度差值；
    根据所述满足第一要求的相似度差值，确定所述任一行对应的动态阈值。
  3. 根据权利要求1或2所述的方法，其中，所述根据所述初始相似度矩阵中各行对应的动态阈值，对所述初始相似度矩阵进行调整，得到参考相似度矩阵，包括：
    将所述初始相似度矩阵第k行包括的相似度中，小于第k行对应的动态阈值的相似度调整为第一数值，并基于各行的调整结果得到所述参考相似度矩阵，k为正整数；
    或者，将所述初始相似度矩阵第k行包括的相似度中，小于所述第k行对应的动态阈值的相似度与第二数值相乘，并基于各行的调整结果得到所述参考相似度矩阵。
  4. 根据权利要求1至3任一所述的方法，其中，所述根据所述参考相似度矩阵确定发声得到所述多个音频片段的音频对象的数目，包括：
    根据多个参考参数，对所述参考相似度矩阵进行处理，得到各个参考参数对应的相似度矩阵；
    根据所述多个参考参数和所述各个参考参数对应的相似度矩阵，确定所述多个音频片段中存在的音频对象的数目。
  5. 根据权利要求4所述的方法，其中，所述根据多个参考参数，对所述参考相似度矩阵进行处理，得到各个参考参数对应的相似度矩阵，包括：
    对于所述多个参考参数中的任一参考参数，根据所述任一参考参数，对所述参考相似度矩阵进行数值调整，得到第一相似度矩阵，所述数值调整用于简化所述参考相似度矩阵；
    对所述第一相似度矩阵进行对称化处理，得到第二相似度矩阵，所述第二相似度矩阵中位于第i行第j列的相似度与位于第j行第i列的相似度相同，所述i和所述j为不大于所述多个音频片段的个数的正整数；
    对所述第二相似度矩阵进行行列扩散，得到第三相似度矩阵，所述第三相似度矩阵用于生成多个音频对象之间的边界；
    对所述第三相似度矩阵进行比例调整，得到第四相似度矩阵，所述比例调整用于将所述 第三相似度矩阵中各行包括的相似度调整在同一个范围内；
    对所述第四相似度矩阵进行对称化处理，得到所述任一参考参数对应的相似度矩阵。
  6. 根据权利要求5所述的方法，其中，所述根据所述任一参考参数，对所述参考相似度矩阵进行数值调整，得到第一相似度矩阵，包括：
    对于所述参考相似度矩阵各行包括的多个相似度，将任一参考参数个满足第三要求的相似度之外的相似度调整为第三数值，得到所述第一相似度矩阵；
    或者，将所述参考相似度矩阵包括的多个相似度中，除任一参考参数个满足第三要求的相似度之外的相似度与第四数值相乘，得到所述第一相似度矩阵。
  7. 根据权利要求5或6所述的方法，其中，所述对所述第一相似度矩阵进行对称化处理，得到第二相似度矩阵，包括：
    确定所述第一相似度矩阵对应的转置矩阵；
    将所述第一相似度矩阵和所述第一相似度矩阵对应的转置矩阵中位于相同位置的相似度相加，得到待调整相似度矩阵；
    对所述待调整相似度矩阵包括的多个相似度进行取半操作，得到所述第二相似度矩阵。
  8. 根据权利要求5或6所述的方法，其中，所述对所述第一相似度矩阵进行对称化处理，得到第二相似度矩阵，包括：
    确定所述第一相似度矩阵中位于所述第i行第j列的相似度，与所述第一相似度矩阵中位于所述第j行第i列的相似度中最大的相似度，将所述最大的相似度作为所述第二相似度矩阵中位于所述第i行第j列和所述第j行第i列的相似度，得到所述第二相似度矩阵。
  9. 根据权利要求5至8任一所述的方法，其中，所述对所述第二相似度矩阵进行行列扩散，得到第三相似度矩阵，包括：
    确定所述第二相似度矩阵对应的转置矩阵；
    根据所述第二相似度矩阵和所述第二相似度矩阵对应的转置矩阵，确定所述第三相似度矩阵，所述第三相似度矩阵中位于第m行第n列的相似度基于所述第二相似度矩阵中位于所述第m行的相似度和所述第二相似度矩阵对应的转置矩阵中位于所述第n列的相似度确定，所述m、所述n为不大于所述多个音频片段的个数的正整数。
  10. 根据权利要求5至9任一所述的方法，其中，所述对所述第三相似度矩阵进行比例调整，得到第四相似度矩阵，包括：
    根据所述第三相似度矩阵中各行包括的多个相似度，确定各行对应的最大相似度；
    将所述第三相似度矩阵中各行包括的多个相似度分别与所述各行对应的最大相似度相除，得到所述第四相似度矩阵。
  11. 根据权利要求4至10任一所述的方法，其中，所述根据所述多个参考参数和所述各个参考参数对应的相似度矩阵，确定所述多个音频片段中存在的音频对象的数目，包括：
    根据所述多个参考参数和所述各个参考参数对应的相似度矩阵，确定所述各个参考参数对应的比例值，所述比例值用于指示所述参考参数对应的相似度矩阵中保留的相似度的数量；
    根据所述各个参考参数对应的比例值，确定所述多个音频片段中存在的音频对象的数目。
  12. 根据权利要求11所述的方法，其中，所述根据所述多个参考参数和所述各个参考参数对应的相似度矩阵，确定所述各个参考参数对应的比例值，包括：
    对于所述多个参考参数中的任一参考参数，对所述任一参考参数对应的相似度矩阵进行 拉普拉斯变换，得到所述任一参考参数对应的拉普拉斯矩阵；
    对所述拉普拉斯矩阵进行奇异值分解，得到多个参考特征值；
    在所述多个参考特征值中确定第二特征值和第一数量个第一特征值，所述第二特征值为所述多个参考特征值中的最大值，所述第一特征值为按照第二顺序对所述多个参考特征值进行排序后满足第二要求的参考特征值；
    确定所述第一数量个第一特征值中相邻的两个第一特征值之间的差值，得到多个特征值差值；
    根据所述第二特征值，对第一特征值差值进行归一化处理，得到归一化之后的特征值差值，所述第一特征值差值为所述多个特征值差值中最大的特征值差值；
    根据所述归一化之后的特征值差值和所述任一参考参数，确定所述任一参考参数对应的比例值。
  13. 根据权利要求11或12所述的方法，其中，所述根据所述各个参考参数对应的比例值，确定所述多个音频片段中存在的音频对象的数目，包括：
    根据所述各个参考参数对应的比例值，在所述多个参考参数中确定第一参数，所述第一参数为所述多个参考参数中对应的比例值最小的参考参数；
    确定所述第一参数对应的多个特征值差值；
    调用第一函数对所述第一参数对应的多个特征值差值进行处理，得到所述多个音频片段中存在的音频对象的数目。
  14. 根据权利要求13所述的方法，其中，所述根据所述音频对象的数目，对所述多个音频片段进行聚类，得到各个音频对象发声得到的音频片段，包括：
    对所述第一参数对应的相似度矩阵进行奇异值分解，得到多个分解特征值；
    在所述多个分解特征值中确定与所述音频对象的数目对应分解特征值；
    确定所述音频对象的数目个分解特征值分别对应的特征向量，并生成分解矩阵，所述分解矩阵的行数为所述音频对象的数目，列数为所述音频片段的数目；
    根据所述分解矩阵，确定所述多个音频片段分别对应的特征向量，所述特征向量用于指示对应的音频片段；
    根据所述音频对象的数目和所述多个音频片段分别对应的特征向量，对所述多个音频片段进行聚类，各个音频对象发声得到的音频片段。
  15. 一种音频处理装置，所述装置包括：
    确定模块，用于确定多个音频片段分别对应的声纹向量，所述声纹向量用于表示所述音频片段对应的声纹特征；
    所述确定模块，还用于根据各个音频片段对应的声纹向量，确定初始相似度矩阵，所述初始相似度矩阵中包括任意两个音频片段对应的声纹向量之间的相似度；
    调整模块，用于根据所述初始相似度矩阵中各行对应的动态阈值，对所述初始相似度矩阵进行调整，得到参考相似度矩阵，所述动态阈值用于对不同相似度之间的相似度差值进行调节；
    所述确定模块，还用于根据所述参考相似度矩阵确定发声得到所述多个音频片段的音频对象的数目；
    聚类模块，用于根据所述音频对象的数目，对所述多个音频片段进行聚类，得到各个音频对象发声得到的音频片段。
  16. 根据权利要求15所述的装置，其中，所述确定模块，还用于对于所述初始相似度矩阵中的任一行，按照第一顺序对所述任一行中位于预设相似度范围内的相似度进行排序，得 到第一排序结果；确定所述第一排序结果中相邻的两个相似度之间的相似度差值，得到多个相似度差值；在所述多个相似度差值中确定满足第一要求的相似度差值；根据所述满足第一要求的相似度差值，确定所述任一行对应的动态阈值。
  17. 根据权利要求15或16所述的装置，所述调整模块，用于将所述初始相似度矩阵第k行包括的相似度中，小于第k行对应的动态阈值的相似度调整为第一数值，并基于各行的调整结果得到所述参考相似度矩阵，k为正整数；或者，将所述初始相似度矩阵第k行包括的相似度中，小于所述第k行对应的动态阈值的相似度与第二数值相乘，并基于各行的调整结果得到所述参考相似度矩阵。
  18. 一种计算机设备，所述计算机设备包括处理器和存储器，所述存储器中存储有至少一条程序代码，所述至少一条程序代码由所述处理器加载并执行，以使所述计算机设备实现如权利要求1至14任一所述的音频处理方法。
  19. 一种计算机可读存储介质，所述计算机可读存储介质中存储有至少一条程序代码，所述至少一条程序代码由处理器加载并执行，以使计算机实现如权利要求1至14任一所述的音频处理方法。
  20. 一种计算机程序产品，包括计算机程序或指令，所述计算机程序或指令被处理器执行时实现如权利要求1至14任一所述的音频处理方法。
PCT/CN2023/114040 2022-09-07 2023-08-21 音频处理方法、装置、设备、可读存储介质及程序产品 Ceased WO2024051481A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP23862178.3A EP4586105A4 (en) 2022-09-07 2023-08-21 AUDIO PROCESSING METHOD AND APPARATUS, DEVICE, READABLE STORAGE MEDIA AND PROGRAMMED PRODUCT
US18/583,688 US20240242722A1 (en) 2022-09-07 2024-02-21 Audio processing method and apparatus, device, readable storage medium, and program product

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211088204.6 2022-09-07
CN202211088204.6A CN115168643B (zh) 2022-09-07 2022-09-07 音频处理方法、装置、设备及计算机可读存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/583,688 Continuation US20240242722A1 (en) 2022-09-07 2024-02-21 Audio processing method and apparatus, device, readable storage medium, and program product

Publications (1)

Publication Number Publication Date
WO2024051481A1 true WO2024051481A1 (zh) 2024-03-14

Family

ID=83480765

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/114040 Ceased WO2024051481A1 (zh) 2022-09-07 2023-08-21 音频处理方法、装置、设备、可读存储介质及程序产品

Country Status (4)

Country Link
US (1) US20240242722A1 (zh)
EP (1) EP4586105A4 (zh)
CN (1) CN115168643B (zh)
WO (1) WO2024051481A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115168643B (zh) * 2022-09-07 2023-04-07 腾讯科技（深圳）有限公司 音频处理方法、装置、设备及计算机可读存储介质
CN115602178A (zh) * 2022-10-18 2023-01-13 锐迪科微电子科技（上海）有限公司（Cn） 声纹识别方法及装置、计算机可读存储介质、终端
CN115861889A (zh) * 2022-12-14 2023-03-28 微梦创科网络科技（中国）有限公司 确定视频相似度的方法、训练方法、装置及存储介质
CN116522240B (zh) * 2023-04-27 2026-03-24 电子科技大学 基于自适应阈值的开集辐射源个体识别方法
CN119964596B (zh) * 2025-01-17 2025-11-21 湖南大学 一种基于视听融合聚类的说话人日志生成方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021072893A1 (zh) * 2019-10-18 2021-04-22 平安科技（深圳）有限公司 一种声纹聚类方法、装置、处理设备以及计算机存储介质
CN113327628A (zh) * 2021-05-27 2021-08-31 北京字节跳动网络技术有限公司 音频处理方法、装置、可读介质和电子设备
CN114446284A (zh) * 2022-02-10 2022-05-06 上海喜马拉雅科技有限公司 说话人日志生成方法、装置、计算机设备及可读存储介质
CN114822558A (zh) * 2022-04-15 2022-07-29 马上消费金融股份有限公司 声纹识别方法、装置、电子设备及存储介质
CN115168643A (zh) * 2022-09-07 2022-10-11 腾讯科技（深圳）有限公司 音频处理方法、装置、设备及计算机可读存储介质

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108040032A (zh) * 2017-11-02 2018-05-15 阿里巴巴集团控股有限公司 一种声纹认证方法、账号注册方法及装置
CN108908377B (zh) * 2018-07-06 2020-06-23 达闼科技（北京）有限公司 说话人识别方法、装置和机器人
CN109360572B (zh) * 2018-11-13 2022-03-11 平安科技（深圳）有限公司 通话分离方法、装置、计算机设备及存储介质
CN110337030B (zh) * 2019-08-08 2020-08-11 腾讯科技（深圳）有限公司 视频播放方法、装置、终端和计算机可读存储介质
CN111866607B (zh) * 2020-07-30 2022-03-11 腾讯科技（深圳）有限公司 视频片段定位方法、装置、计算机设备及存储介质
CN112133319B (zh) * 2020-08-31 2024-09-06 腾讯音乐娱乐科技（深圳）有限公司 音频生成的方法、装置、设备及存储介质
CN111933115B (zh) * 2020-10-12 2021-02-09 腾讯科技（深圳）有限公司 语音识别方法、装置、设备以及存储介质
CN114792522B (zh) * 2021-01-26 2026-01-02 阿里巴巴集团控股有限公司 音频信号处理、会议记录与呈现方法、设备、系统及介质
CN113724739B (zh) * 2021-09-01 2024-06-11 腾讯音乐娱乐科技（深圳）有限公司 检索音频和训练声学模型的方法、终端及存储介质
CN114512135B (zh) * 2022-01-17 2025-02-07 马上消费金融股份有限公司 声纹聚类方法、声纹识别方法、装置及电子设备
CN114937462A (zh) * 2022-05-17 2022-08-23 国网黑龙江省电力有限公司佳木斯供电公司 基于声纹智能诊断高压断路器故障检测方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021072893A1 (zh) * 2019-10-18 2021-04-22 平安科技（深圳）有限公司 一种声纹聚类方法、装置、处理设备以及计算机存储介质
CN113327628A (zh) * 2021-05-27 2021-08-31 北京字节跳动网络技术有限公司 音频处理方法、装置、可读介质和电子设备
CN114446284A (zh) * 2022-02-10 2022-05-06 上海喜马拉雅科技有限公司 说话人日志生成方法、装置、计算机设备及可读存储介质
CN114822558A (zh) * 2022-04-15 2022-07-29 马上消费金融股份有限公司 声纹识别方法、装置、电子设备及存储介质
CN115168643A (zh) * 2022-09-07 2022-10-11 腾讯科技（深圳）有限公司 音频处理方法、装置、设备及计算机可读存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4586105A4 *

Also Published As

Publication number Publication date
CN115168643B (zh) 2023-04-07
CN115168643A (zh) 2022-10-11
US20240242722A1 (en) 2024-07-18
EP4586105A1 (en) 2025-07-16
EP4586105A4 (en) 2025-10-29

Similar Documents

Publication Publication Date Title
WO2024051481A1 (zh) 音频处理方法、装置、设备、可读存储介质及程序产品
CN109299315B (zh) 多媒体资源分类方法、装置、计算机设备及存储介质
CN110544272B (zh) 脸部跟踪方法、装置、计算机设备及存储介质
CN110162604B (zh) 语句生成方法、装置、设备及存储介质
CN110807325B (zh) 谓词识别方法、装置及存储介质
CN113763933B (zh) 语音识别方法、语音识别模型的训练方法、装置和设备
JP7324838B2 (ja) 符号化方法並びにその、装置、機器及びコンピュータプログラム
CN111324699B (zh) 语义匹配的方法、装置、电子设备及存储介质
CN111581958A (zh) 对话状态确定方法、装置、计算机设备及存储介质
CN113569042B (zh) 文本信息分类方法、装置、计算机设备及存储介质
CN111753498B (zh) 文本处理方法、装置、设备及存储介质
CN114996515A (zh) 视频特征提取模型的训练方法、文本生成方法及装置
CN114462580A (zh) 文本识别模型的训练方法、文本识别方法、装置和设备
CN114299306B (zh) 获取图像检索模型的方法、图像检索方法、装置和设备
CN117011571A (zh) 图像分类模型的训练方法、装置及设备
CN111737415B (zh) 实体关系抽取方法、实体关系学习模型的获取方法及设备
CN114328815A (zh) 文本映射模型的处理方法、装置、计算机设备及存储介质
CN114281937A (zh) 嵌套实体识别模型的训练方法、嵌套实体识别方法及装置
CN111597823A (zh) 中心词提取方法、装置、设备及存储介质
HK40076008B (zh) 音频处理方法、装置、设备及计算机可读存储介质
HK40076008A (zh) 音频处理方法、装置、设备及计算机可读存储介质
CN114462540B (zh) 聚类模型的训练方法、聚类方法、装置、设备及存储介质
CN116959057A (zh) 面部图像的处理方法、装置、设备及存储介质
HK40067587B (zh) 特徵提取模型的训练方法、图像处理方法、装置及设备
HK40037362B (zh) 词语的上下位关系确定方法、装置、计算机设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23862178

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023862178

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2023862178

Country of ref document: EP

Effective date: 20250407

WWP Wipo information: published in national office

Ref document number: 2023862178

Country of ref document: EP