Color camera sensors often operate by having a 2×2 pattern of filters placed in front of every block of monochrome pixels in the sensor (often duplicating green to get four colors). This is shown in the figure below, where each filter rejects all but a single frequency range of light. There is smooth fall-off between the color bands (which mimics the human eye receptors around red, green, and blue) so that colors lying on the rainbow between each primary produce smoothly-varying mixtures with wavelength. Interpolation is used to combine the values from neighboring pixels to produce an estimate of all three colors at each pixel.

However, the reconstruction filters could also be made to work with a different set of input colors so long as they are algebraically convertible to red, green, and blue. In particular, a filter set that passed the same bands as the standard filter set but passed two of them rather than one.

The figure above shows such a dual-color filter set, which can be though of as passing “not red”, “not green”, and “not blue” colors. Algebra shows that this set of colors can be converted to red, green, and blue as follows (let X = the dual of blue, Y = the dual of green, and Z be the dual of red):

• 2 * red = X + Y – Z
• 2 * green = X – Y + Z
• 2 * blue = -X + Y + Z

The benefit of using this color space is that it collects twice as much light compared to an RGB filter set — each pixel receives twice the light because it rejects only one color range rather than two.

Luminance as the fourth channel

The fourth sensor in the 2×2 array is redundant in both the RGB and dual color spaces. Often, green is duplicated in this fourth spot because it is the color that the human visual system is most sensitive to.

In the human visual system, high-resolution shape is determined by an object’s luminance (brightness), and it ignores blue in the determination of luminance. In the dual color space, “not blue” is equivalent to luminance and so is the logical choice for repeating in the 2×2 array.

In video encoding, luminance depends on all three channels. To enable a little more light to be captured, the fourth pixel could have no filter (other than the global infrared and ultraviolet blocking filter for the whole sensor), producing a direct measurement of total brightness that gathers all of the light within the visible spectrum.

If this is chosen, the luminance channel will be “blown out” compared to the other three channels, clamping to its maximum value at an exposure and gain that work well for the other three channels. It would provide more details in dimmer regions of the image, forming a type of “high dynamic range” sampling within the image.