CN106295679A - A kind of coloured image light source colour method of estimation based on category correction - Google Patents

Abstract

The invention discloses a kind of coloured image light source colour method of estimation based on category correction, first on the image of one group of known luminaire color, extract the edge feature of image, then learnt by method of least square, obtain the correction matrix between edge feature and light source, to pending test image zooming-out edge feature and it is multiplied with correction matrix again, obtains rough light source and estimate；By the way of finding K width adjacent image at feature space, find a class training image close with pending test characteristics of image afterwards, thus relearn, obtain light source accurately and estimate.The present invention relates to parameter few, and owing to the feature of extraction is simple and negligible amounts, so also having the features such as calculating is simple, speed is fast；Additionally, the present invention is method based on study, so high treating effect, degree of accuracy is high, is very suitable for the occasion that the accuracy of estimation of light source colour requires comparison high.

Description

A Color Estimation Method of Color Image Light Source Based on Classification Correction

technical field

The invention belongs to the technical field of computer vision and image processing, and in particular relates to the design of a method for estimating the color of a color image light source based on classification correction.

Background technique

In a natural environment, the same object will appear in different colors under the illumination of different colors of light. For example, green leaves are yellowish in the morning light, but blue in the evening. The human visual system can resist the color change of the light source, so as to constantly perceive the color of the object, that is, the visual system has color constancy. However, limited by technical conditions, the machine does not have this ability, and the pictures taken by physical equipment, such as cameras, will have serious color cast due to the change of the color of the light source. Therefore, how to accurately estimate and remove the color of the light source in the scene based on the existing image information to obtain the color of the object under the standard white light is particularly important.

Computational color constant is dedicated to solving this problem. Its main purpose is to calculate the color of the unknown light source contained in any image, and then use the calculated light source color to correct the original input image. Displayed under standard white light, the so-called standard image is obtained. Since the standard image removes the influence of the color of the light source, for subsequent computing tasks, such as color-based scene classification, there is no misclassification or misretrieval caused by color cast in image retrieval.

Computational color constancy methods can be divided into two categories: learning-based methods and traditional static methods. The traditional static method extracts simple features from the image for light source estimation. Due to the large estimation error, this kind of method cannot meet the engineering needs very well. Based on this requirement, a learning-based method was born on the basis of the traditional non-learning-based method. The more typical learning-based method is the method proposed by G.D.Finlayson in 2013. Reference: G.D.Finlayson, "Corrected-moment illuminant estimation "in Proc.Comput.Vis.IEEEInt.Conf., 2013, pp.1904–1911, this method finds the relationship between features and light sources through feature extraction and regression. Because this method uses regression means, the accuracy of estimating the light source is relatively high, but this method uses the same correction matrix for all images, resulting in a large error in the estimated light source of some images, so it cannot meet the requirements for estimating the color of the light source. Occasions where high accuracy is required, such as the front end of equipment receiving images for intelligent robots or autonomous driving. Therefore, it is particularly important to implement a method to learn different correction matrices for different images.

Contents of the invention

The purpose of the present invention is to solve the problem that the prior art method for estimating the color of the light source in an image scene cannot meet the high requirements for the accuracy of estimating the color of the light source, and proposes a method for estimating the color of the light source in a color image based on classification correction.

The technical solution of the present invention is: a color image light source color estimation method based on classification correction, comprising the following steps:

S1. Extract the edge features of the training image: take N color images of known light sources as the original training set T, and perform convolution operations with the template G after derivation of the Gaussian distribution to obtain the edge value corresponding to each pixel of the image. Extract edge features to obtain the edge feature matrix M of N training images;

S2. Learning the correction matrix: learning the correction matrix C between the feature matrix M calculated in step S1 and the standard light source L of the N training images by the least square method;

S3. Rough light source estimation: use the method in step S1 to extract the edge features of the test image, and multiply it with the correction matrix C learned in step S2 to obtain a rough light source estimation result L1;

S4. Find the training image corresponding to the test image: remove the light source from the test image and the original training set T respectively, and then use the method in S1 to extract edge features respectively to form a feature space; find features similar to the test image in the feature space The K training images of K are used as a new training set TN;

S5. Accurate light source estimation: Repeat steps S1-S4, each time replace the training set T in step S1 with the new training set TN obtained in step S4, and the number of training images is changed from N to K accordingly, until step Until the TN obtained in S4 is the same as the TN obtained in step S4 in the last operation, the light source estimation result L1 obtained in step S3 in the last operation is taken as the final light source estimation result.

Further, the template G after derivation of the Gaussian distribution in step S1 is a Gaussian gradient operator.

Further, the calculation formula for extracting edge features in step S1 is:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\mathrm{<span\; class="notranslate">\; z</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; z</span>}}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

In the formula, R _i , G _i , and B _i represent the edge values of each pixel in the three channels of R, G, and B respectively, N ₁ represents the number of pixels in the image, and M _xyz is different x, y, z The values of the corresponding edge features, x, y, z are all combinations satisfying x≥0, y≥0, z≥0 and x+y+z=3.

Further, the value range of K in step S4 is

Further, step S4 specifically includes the following sub-steps:

S41. Remove the standard light source L from the original N training images, and extract edge features using the method in step S1;

S42, removing the roughly estimated light source L1 in step S3 from the test image, and extracting edge features using the method in step S1, forming a feature space together with the edge features extracted from the N training images in step S41;

S43. Find K images in the feature space that are closest to the feature distance of the test image, and use it as a new training image set TN for the test image.

Further, the feature distance in step S43 is Euclidean distance.

The beneficial effects of the present invention are: the present invention first extracts the edge features of the image on a group of images with known light source colors, and then learns through the least square method to obtain the correction matrix between the edge features and the light source, and then the test to be processed The image extracts edge features and multiplies them with the correction matrix to obtain a rough light source estimate; then find a class of training images that are similar to the characteristics of the test image to be processed by searching for K adjacent images in the feature space, so as to relearn and obtain accurate Light source estimation. Since the distance between the test image to be processed and the training image in the feature space is different, properly adjusting the value of K corresponding to the number of training images can obtain better results suitable for different types of training images, where K is the only parameter. The present invention involves few parameters (only one parameter K), and because the extracted features are simple and small in number, it also has the characteristics of simple calculation and fast speed; in addition, the present invention is based on a learning method, so the processing effect is good, With high accuracy, it is very suitable for occasions where the estimation accuracy of the color of the light source is relatively high, such as being built in the front end of an intelligent robot or an automatic driving image receiving device, etc.

Description of drawings

FIG. 1 is a flow chart of a color image light source color estimation method based on classification correction provided by the present invention.

Fig. 2 is the test image tools_ph-ulm.GIF to be processed in Embodiment 1 of the present invention.

FIG. 3 is a schematic diagram of error values between the estimated light source and the real light source in each step of Embodiment 1 of the present invention.

FIG. 4 is a schematic diagram of a comparison between the final light source estimation result and the real light source according to Embodiment 1 of the present invention.

FIG. 5 is a schematic diagram of the result of performing tone correction on the original test image by using the color value of the light source calculated in step S5 according to Embodiment 2 of the present invention.

detailed description

Embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

The present invention provides a color image light source color estimation method based on classification correction, as shown in Figure 1, comprising the following steps:

S1. Extract the edge features of the training image: take N color images of known light sources as the original training set T, and perform convolution operations with the template G after derivation of the Gaussian distribution to obtain the edge value corresponding to each pixel of the image. Extract the edge features to obtain the edge feature matrix M of N training images.

Among them, the template G after Gaussian distribution derivation is a Gaussian gradient operator.

The calculation formula for extracting edge features is:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\mathrm{<span\; class="notranslate">\; z</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; z</span>}}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

In the formula, R _i , G _i , and B _i represent the edge values of each pixel in the three channels of R, G, and B respectively, N ₁ represents the number of pixels in the image, and M _xyz is different x, y, z The values of the corresponding edge features, x, y, z are all combinations that satisfy x≥0, y≥0, z≥0 and x+y+z=3, the total number of combinations is 19, so 19 can be obtained here edge features.

S2. Learning the correction matrix: learning the correction matrix C between the feature matrix M calculated in step S1 and the standard light source L of the N training images by the least square method.

S3. Rough light source estimation: use the method in step S1 to extract the edge features of the test image, and multiply it with the correction matrix C learned in step S2 to obtain a rough light source estimation result L1.

S4. Find the training image corresponding to the test image: remove the light source from the test image and the original training set T respectively, and then use the method in S1 to extract edge features respectively to form a feature space; find features similar to the test image in the feature space K training images of K (the value range of K is ) as a new training set TN.

This step specifically includes the following sub-steps:

S41. Remove the standard light source L from the original N training images, and extract edge features using the method in step S1.

S42. Remove the roughly estimated light source L1 in step S3 from the test image, and extract edge features using the method in step S1, and form a feature space together with the edge features extracted from the N training images in step S41.

S43. Find K images in the feature space that are closest to the feature distance of the test image, and use it as a new training image set TN for the test image.

Wherein, the feature distance in step S43 is Euclidean distance.

S5. Accurate light source estimation: Repeat steps S1-S4, each time replace the training set T in step S1 with the new training set TN obtained in step S4, and the number of training images is changed from N to K accordingly, until step Until the TN obtained in S4 is the same as the TN obtained in step S4 in the last operation, the light source estimation result L1 obtained in step S3 in the last operation is taken as the final light source estimation result.

The final light source estimation result L1 of the image calculated after step S5 can be directly used in subsequent computer vision applications. For example, by dividing the component of each color channel of the input original color image by L1, the color of the light source in the color image can be removed. the goal of. In addition, the white balance and color correction of the image also need to use the final light source estimation result L1 calculated in S5.

A method for estimating the color image light source color based on classification correction provided by the present invention will be further described in a specific embodiment below:

Embodiment one:

Download all the images (321 in total) of the currently internationally recognized image library SFU object for estimating scene light source colors and their corresponding real light source colors (standard light source) L. The image size is 468×637, and the first 214 images in the image library images as the training set images, select one of the remaining images tools_ph-ulm.GIF (as shown in Figure 2) as the test image to be processed for testing, all images have not undergone any preprocessing of the camera itself (such as tone correction , gamma value correction). Then the detailed steps of the present invention are as follows:

S1. Extract the edge features of the training image: use 214 color images of known light sources as the original training set T, and perform convolution operations with the template G (Gaussian gradient operator) after derivation of the Gaussian distribution to obtain each pixel of the image The edge value corresponding to the point, and then extract the 19-dimensional edge features respectively, and finally obtain the edge feature matrix M of the training set image with a size of 214*19.

S2. Learning the correction matrix: learn the correction matrix between the feature matrix M calculated in step S1 and the standard light source L of the 214 training images by the least square method, and obtain a correction matrix C with a size of 19*3:

C＝[-150.0689，-30.1462，-21.5186；-96.5582，-196.1642，-348.5298；52.6551，76.4461，115.5982；-200.5289，-240.3650，-179.6495；-79.6311，72.4539，125.1126；-56.1276，-130.2963，- 226.1518；683.9180，552.9035，366.8781；214.1444，-15.5379，-52.8198；-149.3407，138.0260，397.1888；154.6218，240.3336，128.2161；156.5752，-50.6503，69.4182；22.7103，90.3730，274.5781；-65.9786，-384.7642，-66.2556 ；-112.7044，-104.0913，-12.8868；-349.7427，81.5115，-215.8972；-79.0109，-48.0727，-32.2072；-98.2723，-22.7039，-51.2091；108.6481，-52.0896，-265.9989；172.6056，171.2726，95.1991] .

S3. Rough light source estimation: use the method in step S1 to extract the 19-dimensional edge features of the test image, and obtain the edge feature matrix M1 of the test image with a size of 1*19:

M1=[0.0002, 0.0004, 0.0002, 0.0014, 0.0017, 0.0012, 0.0015, 0.0013, 0.0014, 0.0036, 0.0040, 0.0031, 0.0037, 0.0034, 0.0039, 0.0035, 0.00332, 0.0]

Then multiply M1 with the correction matrix C learned in step S2 to obtain a rough light source estimation result L1=[0.1985, 0.2151, 0.2360].

S4. Find the training image corresponding to the test image: remove the light source from the test image and the original training set T with 214 images, and then use the method in S1 to extract 19-dimensional edge features to form a feature space. In the feature space, find K training images with similar features to the test image, so as to obtain a class of images with similar features, and use these K images as a new training set TN. In the embodiment of the present invention, K=100 is selected.

This step specifically includes the following sub-steps:

S41. Remove the standard light source L from the original 214 training images, and use the method in step S1 to extract 19-dimensional edge features to obtain a feature matrix M0 with a size of 214*19.

S42. Remove the roughly estimated light source L1 in step S3 from the test image, and use the method in step S1 to extract edge features to obtain a feature matrix M2 with a size of 1*19:

M2=[0.0060, 0.0089, 0.0038, 0.0366, 0.0371, 0.0224, 0.0365, 0.0282, 0.0285, 0.0937, 0.0900, 0.0581, 0.0921, 0.0790, 0.0909, 0.0768, 0.06773, 0.0]

Then M2 and the edge feature M0 extracted from the 214 training images in step S41 form a feature space together.

S43. Find 100 images in the feature space that are closest to the feature distance of the test image, and use it as a new training image set TN for the test image.

S5. Accurate light source estimation: Repeat steps S1-S4, each time replace the training set T in step S1 with the new training set TN obtained in step S4, and the number of training images is changed from N to K accordingly, until step Until the TN obtained in S4 is the same as the TN obtained in step S4 in the last operation, the light source estimation result L1 obtained in step S3 in the last operation is taken as the final light source estimation result.

In the embodiment of the present invention, in order to save time, the operation can be repeated twice. The light source estimation result obtained after repeating the operation once is L1 = [0.3412, 0.3591, 0.3168], and the light source estimation result obtained after repeating the operation twice is L1 = [0.3312, 0.3365, 0.3430]. The light source estimation result L1=[0.3312, 0.3365, 0.3430] obtained after two executions is taken as the final light source estimation result.

As shown in Figure 3, the first column represents the angular error value between the roughly estimated light source and the real light source in step S3, and the second column is the angular error between the estimated light source and the real light source after repeated operations in step S5 value, the third column is the angle error value between the estimated light source and the real light source after repeating the operation twice in step S5. The broken line connecting the three columns reflects the downward trend of the estimation error, indicating that the estimated light source is getting more and more accurate.

As shown in Figure 4, the direction of the response of the red and green components in the three primary color spaces finally calculated in step S5 and the direction of the response of the red and green components of the real light source, Figure 4 shows that the response value calculated by step S5 is consistent with the real scene light source The color information is close.

The following is a simple demonstration of the light source estimation result finally obtained by the present invention, taking image tone correction as an example in a practical application, using a specific embodiment:

Embodiment two:

The pixel values of each color component of the original input image are respectively corrected by using the light source color values under each color component calculated in step S5. Taking a pixel point (0.335,0.538,0.601) of the test image input in step S3 as an example, the corrected result is (0.335/0.3312,0.538/0.3365,0.601/0.3430)=(1.0115,1.5988,1.7522), After normalization, it becomes (0.2319, 0.3665, 0.4016), and then multiply the corrected value by the standard white light coefficient Obtain (0.1339, 0.2116, 0.2319) as the pixel value of the final output corrected image, do similar calculations for other pixels of the original input image, and finally obtain the corrected color image, as shown in Figure 5.

Those skilled in the art will appreciate that the embodiments described here are to help readers understand the principles of the present invention, and it should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical revelations disclosed in the present invention without departing from the essence of the present invention, and these modifications and combinations are still within the protection scope of the present invention.

Claims (6)

1. a coloured image light source colour method of estimation based on category correction, it is characterised in that comprise the following steps:

S1, the edge feature of extraction training image: using the coloured image of N width known luminaire as original training set T, respectively with height Template G after this distribution derivation does convolution algorithm, obtains the marginal value that each pixel of image is corresponding, extracts edge feature, Edge feature matrix M to N width training image；

S2, learning correction matrix: by method of least square, learn by step S1 calculated eigenmatrix M and N width training figure Correction matrix C between standard light source L of picture；

S3, rough light source are estimated: use the method in step S1 to extract the edge feature of test image, learn with step S2 To correction matrix C be multiplied, obtain rough light source estimated result L1；

S4, find the training image corresponding with testing image: test image is removed light source respectively with original training set T, then Use the method in S1 to extract edge feature respectively, form feature space；Feature space is found out and tests characteristics of image phase Near K width training image, as new training set TN；

S5, accurately light source estimate: repeat step S1-S4, training set T in step S1 replaced with in step S4 every time and obtain New training set TN, training image number is also become K from N accordingly, until in step S4 the TN that obtains with in last operation Till TN that step S4 obtains is identical, the light source estimated result L1 that step S3 in last operation is obtained is as final light source Estimated result.

Coloured image light source colour method of estimation based on category correction the most according to claim 1, it is characterised in that institute Stating in step S1 template G after Gauss distribution derivation is Gauss gradient operator.

Coloured image light source colour method of estimation based on category correction the most according to claim 1, it is characterised in that institute State and step S1 is extracted the computing formula of edge feature be:

$M_{xyz}={\left(\frac{{\mathrm{\&Sigma;}}_{i=1}^{N_1}{R}_i^x{G}_i^y{B}_i^z}{N_1}\right)}^{1/(x+y+z)}---\left(1\right)$

R in formula_i、G_i、B_iRepresent each pixel marginal value at tri-passages of R, G, B, N respectively₁Represent pixel in image Number, M_xyzFor different x, the value of edge feature corresponding under y, z, x, y, z are to meet x >=0, y >=0, z >=0 and x+y+z=3's All combinations.

Coloured image light source colour method of estimation based on category correction the most according to claim 1, it is characterised in that institute Stating the span of K in step S4 is

Coloured image light source colour method of estimation based on category correction the most according to claim 1, it is characterised in that institute State step S4 specifically include following step by step:

S41, original N width training image is removed standard light source L, and use method in step S1 to extract edge feature；

S42, to the light source L1 of rough estimate in test image removal step S3, and it is special to use the method in step S1 to extract edge Levying, the edge feature extracted with N width training image in step S41 is collectively forming feature space；

S43, in feature space, find out and test the characteristics of image the most close K width image of distance, as the new instruction of test image Practice image collection TN.

Coloured image light source colour method of estimation based on category correction the most according to claim 5, it is characterised in that institute Stating the characteristic distance in step S43 is Euclidean distance.