Color normalization

Color normalization is a topic in computer vision concerned with artificial color vision and object recognition. In general, the distribution of color values in an image depends on the illumination, which may vary depending on lighting conditions, cameras, and other factors. Color normalization allows for object recognition techniques based on color to compensate for these variations.

Color constancy is a feature of the human internal model of perception, which provides humans with the ability to assign a relatively constant color to objects even under different illumination conditions. This is helpful for object recognition as well as identification of light sources in an environment. For example, humans see an object approximately as the same color when the sun is bright or when the sun is dim.

Color normalization has been used for object recognition on color images in the field of robotics, bioinformatics and general artificial intelligence, when it is important to remove all intensity values from the image while preserving color values. One example is in case of a scene shot by a surveillance camera over the day, where it is important to remove shadows or lighting changes on same color pixels and recognize the people that passed.^[1] Another example is automated screening tools used for the detection of diabetic retinopathy^[2] as well as molecular diagnosis of cancer states,^[3] where it is important to include color information during classification.

The main issue about certain applications of color normalization is that the result looks unnatural or too distant from the original colors.^[4] In cases where there is a subtle variation between important aspects, this can be problematic. More specifically, the side effect can be that pixels become divergent and not reflect the actual color value of the image. A way of combating this issue is to use color normalization in combination with thresholding to correctly and consistently segment a colored image.^[5]

There is a vast array of different transformations and algorithms for achieving color normalization and a limited list is presented here. The performance of an algorithm is dependent on the task and one algorithm which performs better than another in one task might perform worse in another (no free lunch theorem). Additionally, the choice of the algorithm depends on the preferences of the user for the end-result, e.g. they may want a more natural-looking color image.

The grey world normalization makes the assumption that changes in the lighting spectrum can be modelled by three constant factors applied to the red, green and blue channels of color.^[6] More specifically, a change in illuminated color can be modelled as a scaling α, β and γ in the R, G and B color channels and as such the grey world algorithm is invariant to illumination color variations. Therefore, a constancy solution can be achieved by dividing each color channel by its average value as shown in the following formula:

${\displaystyle \left(\alpha R,\beta G,\gamma B\right)\rightarrow \left({\frac {\alpha R}{{\frac {\alpha }{n}}\sum _{i}R}},{\frac {\beta G}{{\frac {\beta }{n}}\sum _{i}G}},{\frac {\gamma B}{{\frac {\gamma }{n}}\sum _{i}B}}\right)}$

As mentioned above, grey world color normalization is invariant to illuminated color variations α, β and γ, however it has one important problem: it does not account for all variations of illumination intensity and it is not dynamic; when new objects appear in the scene it fails.^[6] To solve this problem there are several variants of the grey world algorithm.^[6] Additionally there is an iterative variation of the grey world normalization, however it was not found to perform significantly better.^[7]

Histogram equalization is a non-linear transform which maintains pixel rank and is capable of normalizing for any monotonically increasing color transform function. It is considered to be a more powerful normalization transformation than the grey world method. The results of histogram equalization tend to have an exaggerated blue channel and look unnatural, due to the fact that in most images the distribution of the pixel values is usually more similar to a Gaussian distribution, rather than uniform.^[5]

Histogram specification transforms the red, green and blue histograms to match the shapes of three specific histograms, rather than simply equalizing them. It refers to a class of image transforms which aims to obtain images of which the histograms have a desired shape.^[2] As specified, firstly it is necessary to convert the image so that it has a particular histogram. Assume an image x. The following formula is the equalization transform of this image:

${\displaystyle y=f(x)=\int \limits _{0}^{x}p_{x}(u)du}$

Then assume wanted image z. The equalization transform of this image is:

${\displaystyle y'=g(z)=\int \limits _{0}^{z}p_{z}(u)du}$

Of course ${\displaystyle p_{z}(u)}$ is the histogram of the output image. The formula to find the inverse of the above transform is:

${\displaystyle z=g^{-1}(y')}$

Therefore, since images y and y' have the same equalized histogram they are actually the same image, meaning y = y' and the transform from the given image x to the wanted image z is:

${\displaystyle z=g^{-1}(y')=g^{-1}(y)=g^{-1}(f(x))}$

Histogram specification has the advantage of producing more realistic looking images,^[8] as it does not exaggerate the blue channel like histogram equalization.

The comprehensive color normalization is shown to increase localization and object classification results in combination with color indexing.^[7] It is an iterative algorithm which works in two stages. The first stage is to use the red, green and blue color space with the intensity normalized, to normalize each pixel. The second stage is to normalize each color channel separately, so that the sum of the color components is equal to one third of the number of pixels. The iterations continue until convergence, meaning no additional changes. Formally:

Normalize the color image

${\displaystyle f^{(t)}=[f_{ij}^{(t)}]_{i=1...N,j=1...M}}$

which consists of color vectors

${\displaystyle f_{ij}^{(t)}=(r_{ij}^{(t)},g_{ij}^{(t)},b_{ij}^{(t)})^{T}.}$

For the first step explained above, compute:

${\displaystyle S_{ij}:=r_{ij}^{(t)}+g_{ij}^{(t)}+b_{ij}^{(t)}}$

which leads to

${\displaystyle r_{ij}^{(t+1)}={\frac {r_{ij}^{(t)}}{S_{ij}}},g_{ij}^{(t+1)}={\frac {g_{ij}^{(t)}}{S_{ij}}}}$

and

${\displaystyle b_{ij}^{(t+1)}={\frac {b_{ij}^{(t)}}{S_{ij}}}.}$

For the second step explained above, compute:

${\displaystyle r'={\frac {3}{NM}}\sum _{i=1}^{N}\sum _{j=1}^{M}r_{ij}^{(t+1)}}$

and normalize

${\displaystyle r_{ij}^{(t+2)}={\frac {r_{ij}^{(t+1)}}{r'}}.}$

Of course the same process is done for b' and g'. Then these two steps are repeated until the changes between iteration t and t+2 are less than some set threshold.

Comprehensive color normalization, just like the histogram equalization method previously mentioned, produces results that may look less natural due to the reduction in the number of color values.^[4]

• L.Csink; D. Paulus; U. Ahlrichs; B. Heigl (September 24, 1990). Color Normalization and Object Localization (PDF) (Report).

• Mark A. Rubin; Maciej P. Zerkowski; Robest L. Camp; Rainer Kuefer; Matthias D. Hofer; Arul M. Chinnaiyan; David L. Rimm (March 2004). "Quantitative Determination of Expression of the Prostate Cancer Protein a-Methylacyl-CoA Racemase Using Automated Quantitative Analysis (AQUA)". The American Journal of Pathology. 164 (3): 831–840. doi:10.1016/s0002-9440(10)63171-9. PMC 1613273. PMID 14982837.

• Maria Vanrell; Felipe Lumbreras; Albert Pujol; Ramon Baldrich; Josep Llados; J.J. Villanueva (October 7, 2001). "Colour normalisation based on background information". Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205). Vol. 1. Thessaloniki, Greece. pp. 874–877. doi:10.1109/icip.2001.959185. ISBN 978-0-7803-6725-8. S2CID 206810375. INSPEC 7210999. {{cite book}}: |journal= ignored (help)CS1 maint: location missing publisher (link)

• Burger, Wilhelm; Mark J. Burge (2008). Digital Image Processing: An algorithmic introduction using Java. Springer. ISBN 978-1846283796.

• G.D. Finlayson; B. Shiele; J.L. Crowley (1998). "Comprehensive Colour Image Normalization" (PDF). Burkhard and Neumann: 475–490. INSPEC 7210999. Archived from the original (PDF) on March 4, 2016. Retrieved March 10, 2012.

• A. Osareh; M. Mirmehdi; B. Thomas; et al. (2002). Classification and localisation of diabetic-related eye disease. Eccv '02. pp. 502–516. ISBN 9783540437482. {{cite book}}: |journal= ignored (help)

• Juan Sanchez; Xavier Binefa (200). Laurini, Robert (ed.). Color Normalization for Digital Video Processing. Lecture Notes in Computer Science. Vol. 2929. Springer. pp. 125–139. doi:10.1007/3-540-40053-2_17. ISBN 978-3-540-41177-2.
• Jose M. Buenaposada; Luis Baumela. Variations of Grey World for face tracking (PDF) (Report).
• Keith A. Goatman; A. David Whitwam; A. Manivannan; John A. Olson; Peter F. Sharp. Colour normalisation of retinal images (PDF) (Report).

• Image Enhancement by Contrast Transform Archived January 18, 2012, at the Wayback Machine
• Color Constancy MIT