WHITE PAPER Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision

Abstract
This paper addresses the requirements for wide dynamic range cameras in automotive machine vision applications. The paper describes the architecture of a CMOS image sensor pixel and explores mechanisms for overcoming the effects of noise and saturation to achieve wide dynamic range. The linear response curve of a standard pixel is used to illustrate the effect of integration time on dynamic range. Alternative methods are presented for expanding dynamic range of image sensors, including multiple integration and logarithmic response. Each method is shown to have signicant disadvantages. Autobrite proprietary technology from Melexis is described. An explanation of variable height/multiple reset and barrier voltage demonstrates how Autobrite offers superior interscene and intrascene dynamic range. A description of the control mechanism shows how Autobrite provides adaptive and programmable wide dynamic range. The paper concludes that only cameras equipped with Autobrite match the dynamic range of the image sensor to both the scene and the application. Autobrite-enabled cameras offer superior wide dynamic range performance for capturing complete and accurate visual information in driver assistance systems. the resulting image will fail to capture details that the automotive machine vision system requires. Most of these details cannot be recovered through post-processing. Figure 1 lists typical automotive imaging applications for machine vision. In each of these applications, wide dynamic range is essential for capturing all available details for further electronic processing by the driver assistance system. For example, a blind spot monitoring system compares video frames and alerts the driver when a vehicle is moving into the monitored area. In order to detect moving vehicles but ignore stationary items such as road barriers, the processor must receive accurate and complete digital information from the camera in both daylight and darkness.

Contents
1. Introduction 2. The CMOS Image Sensor 3. Dynamic Range Overview 4. Dynamic Range of Cameras Using Linear Image Sensors 5. Methods for Expanding Dynamic Range 6. The Autobrite Solution for Wide Dynamic Range 7. Autobrite in Automotive Applications 8. Conclusion

1. Introduction
Machine vision applications in the automotive industry face critical imaging challenges. Driver assistance systems must process and respond quickly and accurately to visual details from uncontrolled environments. This ability relies on obtaining complete and accurate video information from a camera that adapts to widely varying light levels. The ratio of the highest to the lowest intensity of light in a scene is known as the dynamic range. Conditions such as approaching headlights, glare from other vehicles, tunnel entrances and exits, and rising or setting sun produce extreme variations of illumination within a scene. If the dynamic range of a camera is not sufciently wide to accommodate the dynamic range of a scene,

Driver assistance systems rely on complete and accurate video information from a camera that adapts to widely varying light levels.

WHITE PAPER
Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision

Figure 1. Applications of Machine Vision in Driver Assistance

Figure 2 compares two images captured in real-world automotive lighting conditions. The image on the left, captured using a camera without wide dynamic range, has lost much of the detail in the brightest and darkest areas. The image on the right, captured with a wide dynamic range camera, accommodates the extremes of lighting and includes all of the important details in the scene. In applications such as blind spot monitoring or lane departure warning, the information provided in the image on the right is critical. Only the image on the right provides adequate detail for the machine vision system to analyze and inform the driver about safe lane changing conditions. An automotive machine vision system using the image on the left has a greater chance of failing.

Figure 2. Images captured without (left) and with (right) Wide Dynamic Range
2

WHITE PAPER
Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision
2.The CMOS Image Sensor The image sensor determines the cameras ability to accommodate the dynamic range of a scene.
The camera is the front end of any machine vision system. The image sensor is the critical component that determines the cameras ability to accommodate the dynamic range of a scene. This section introduces a basic CMOS pixel architecture and explains charge accumulation, saturation, and the reset function. The image sensor contains an array of pixels, each with a photodiode that converts light (photons) to charge (electrons). Figure 3 shows a typical structure of an acti ve CMOS pixel. The charge collected in the pixel is proportional to the intensity of light that strikes the photodiode. Charge is accumulated over time and then sent to the output circuit at the end of the integration period.
V uc

b. During the integration period: Integration starts by lowering the gate voltage at M1, bringing the gate to ground. M1 stops conducting and the starting voltage is stored on the photodiode. As photons strike the photodiode, photo current is integrated onto the photodiode capacitance, reducing the stored voltage. c. At end of the integration period: The nal voltage, representing the amount of light impinging on the photodiode, is buffered by M2 and selectively read to an output device by turning on M3. Before the next integration period starts, the reset voltage is applied, causing electrons to be spilled off or removed from the photodiode. Figure 4 illustrates the function of the pixel reset. In Figure 4a (left), the reset transistor is off during the integration period, while light strikes the photodiode and electrons accumulate. In Figure 4b (center), the reset transistor is turned on and electrons are extracted from the photodiode out to the power supply. In Figure 4c (right), the photodiode is empty after the reset, except for residual electrons. During the integration period, charge accumulates linearly on the photodiode capacitance as a function of time. If enough charge accumulates before the reset
Light

is applied, the potential reaches its minimum value and the pixel is saturated. Also referred to as the ll or full-well capacity of the pixel, the saturation point is measured in electrons and is a function of the photodiode size. Figures 5 and 6 illustrate charge accumulation and pixel saturation. In Figure 5, charge accumulates linearly over time, as indicated by the sloped curve, until the pixel saturates. At saturation, the slope becomes horizontal and no further charge accumulates. Figure 6 shows the photodiode at saturation. Any additional photons striking the photodiode are not detected until the pixel is reset. Therefore, it is impossible to differentiate between illumination levels that exceed the saturation level.

Charge

Saturation

R es et

M1 M2

Illumination
Figure 5. Charge Accumulation and Pixel Saturation

P hotodiode M3 R ow S elect Output

Figure 3. CMOS pixel architecture

The CMOS pixel functions as follows: a. Before the start of the integration period: Reset voltage VReset (typically the positive power supply voltage Vuc) is applied to the gate of reset transistor M1. Current ows through M1 until the photodiode reaches a voltage near the power supply voltage Vuc.

Reset off

Reset on

e-

e-

Vuc

(a) Photodiode during integration

Vuc

(b) Photodiode during reset

Vuc

(c) Photodiode after reset

Figure 4. Pixel Reset

3

WHITE PAPER
Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision
Light

e-

The dynamic range of a camera determines its ability to capture the dynamic range of a scene. The uncontrolled lighting that is typical of automotive environments,outside, inside, or at night presents a signicant challenge. Scenes with some regions that are very bright and other regions that are very dim pose the greatest difculty. If the actual scene has a greater dynamic range than a camera can capture, the resultant images will lack detail in the dark or bright areas. In the darkest areas, where the signal is lower than the noise, information such as trees in shadows or people in dark clothing is not resolved. In the brightest areas, where the signal is greater than saturation, information such as reections off oncoming vehicles or glare off a roadway is washed out. Either situation results in incomplete digital detail for the machine vision system to process. Figure 7 lists the luminance, measured in candelas per square meter, for typical scenes in an automotive environment. In a scene that includes, for example, a bright object in sunlight (105 cd/m2) as well as dark colors in shadows (10-1 cd/m2), the dynamic range is calculated as:
20 Log 10 5 120 dB 10 1

To accurately capture the details in a scene with such extremes of lighting, a camera must provide a dynamic range of at least 120 dB. This requirement is typical of scenes in automotive machine vision applications. For comparison, cameras built with traditional CMOS image sensors provide dynamic range of around 60 dB.

4. Dynamic Range of Cameras Using Linear Image Sensors An automotive system designer using a camera with a linear image sensor must choose between detecting the bright or dark details, but cannot detect both.
This section describes a typical image sensor with linear response, in which the stored charge is directly proportional to the amount of illumination. Figure 8 shows a response curve for a linear image sensor. The slope of the curve represents the image sensors responsivity, the change in signal due to change in illumination
Signal lower than noise Dyanmic range Signal higher than saturation Saturation

Vuc
Figure 6. Saturated Pixel

Any illumination level that results in saturation will produce the same result in the image: a white pixel. For automotive applications that require precise details even in bright areas of an image, this is suboptimalthus the need for a wide dynamic range imaging solution.

3. Dynamic Range Overview The dynamic range of a camera is the ratio of the maximum to the minimum detectable signal of the image sensor, limited by saturation and the noise oor.
This section introduces dynamic range of a camera, the ratio of the maximum to the minimum detectable signal of the image sensor. The maximum signal is limited by saturation, while the minimum signal is limited by the noise oor. Noise oor represents the level of noise inherent in all pixels, resulting from process technology limitations such as dark current and circuit design factors such as reset noise. Dynamic range is expressed in decibel units (dB) according to the equation:
Dynamic Range 20 Log Signal ( saturation) Signal (noise)

Charge

Noise Floor

Illumination

Figure 8. Linear response curve

Figure 7 Luminance Values in Outdoor Scenes

4

At the lowest illumination level, all cameras lose some information because the signal falls below the noise oor. Figure 8 shows this response as the gray part of the linear curve. An image sensors response to the illumination must rise above the noise oor before the camera can distinguish between noise and the signal that results from

WHITE PAPER
Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision
photons striking the photodiode. In other words, if the signal-to-noise ratio is too low, details in the dark areas are not distinguishable from noise. If details are important at low levels of brightness, the noise oor is of greatest concern. At a high illumination level, a standard camera loses information due to saturation. Once a pixel saturates, no further information about its brightness is available. Even if the illumination continues to increase, there is no difference in the output signal, as shown in Figure 8 where the slope of the curve becomes horizontal. The dynamic range is the part of the curve between the noise oor and saturation. To capture the maximum amount of information, the response curve must cross the noise oor quickly and not reach the saturation point until the highest possible illumination. Linear image sensors typically use integration time to increase the dynamic range. Integration time is the duration over which the pixel accumulates charge due to exposure to illumination. Figure 9 compares the linear response curve for two integration periods.
Long Integration Time Short Integration Time

Figure 9b (bottom diagram) shows that the shallow slope of the short integration time response curve does not reach saturation until much higher illumination. In other words, good detail can be captured from bright objects without washing out. However, the short integration time requires more illumination to rise above the noise oor. When an image sensor fails to capture enough photons to provide detail in the darker regions, darker pixels will blend together and information about the original image is lost.
(a) Long Integration

Table 1 summarizes the observations about the response curves in Figure 9. As Table 1 shows, long integration requires little light to cross the noise oor and capture information in the dark areas, but saturates quickly. Short integration time does not reach saturation until high illumination, but also requires high illumination to cross the noise oor. An automotive system designer using a camera with a linear image sensor must choose between detecting the bright or dark details, but cannot detect both. In general, integration time is useful for expanding the interscene dynamic range in cameras using linear image sensors. If one scene is predominantly dark and the next scene is predominantly bright, a long integration time will work for the rst image and a short integration for the second. However, integration time has limited usefulness for expanding intrascene dynamic range. If extremes of light and dark exist in the same scene, a short integration time will lose details in the dark while a long integration will wash out the bright areas. Automotive scenes often have extremes of lighting at the same time, requiring high intrascene dynamic range. A camera with a linear image sensor, therefore, is inadequate for automotive applications because it cannot detect both bright and dark details in a single scene. An alternative solution is needed that captures both.

Charge

Saturation

Noise Floor
Crosses noise oor at low illumination Reaches saturation at low illumination

Illumination

(b) Short Integration

Charge

Saturation

Noise Floor
Crosses noise oor at high illumination Reaches saturation at high illumination

Figure 9a (top diagram) shows that the steep slope of a long integration time enables the response curve to rise quickly above the noise oor. In other words, relatively little light is required to produce a detectable response and good detail can be captured in the dark regions of a scene. However, the steep slope of the long integration time response curve reaches saturation quickly. The image sensor reaches its maximum output in relatively low light, resulting in large portions of the image saturating and appearing to be uniformly white.

Illumination

Figure 9 Comparison of linear response curves for long and short integration time

Table 1. Effects of integration time on linear response curve Integration Time Short Long Illumination Level Required to Reach Noise Floor High Low Illumination Level at which Saturation Occurs High Low

WHITE PAPER
Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision
5. Methods for Expanding Dynamic Range Methods for expanding dynamic range should meet the criteria for an ideal automotive camera, including adaptablility and programmability.
As shown in the previous section, traditional cameras with linear image sensors have a number of failings. Current linear image sensor technology does not capture as much dynamic range as might be available in a scene. While integration time is an effective tool for expanding the interscene dynamic range (from scene to scene), it is not useful for expanding the intrascene dynamic range (within a scene). This section describes several methods for expanding dynamic range and evaluates each method against criteria for an ideal automotive camera. The ideal automotive camera uses an image sensor that produces a wide dynamic range response curve with the following characteristics: In low light, the response curve is linear with a steep slope to maximize responsivity and ensure that the signal exceeds the noise oor at the lowest possible illumination. In bright light, the response curve is nonlinear to expand the dynamic range as far as possible before reaching saturation. Figure 10 proposes an ideal response curve. However, having an image sensor that produces the curve shown in Figure 10 is not enough to dene the ideal automotive camera. For example, a camera is impractical if it is prohibitively expensive to manufacture. Furthermore, a camera should be exible to change the response curve according to the conditions.
Saturation

An optimal method for achieving wide dynamic range should meet all of the criteria summarized in Table 2. The following are just a few examples of the methods that have been developed for expanding dynamic range. Each example uses a different approach to achieving the criteria shown in Table 2.
Multiple Integration

Linear curve, steep slope at low illumination

Non-linear curve, shallow slope at high illumination

A common technique for providing wide dynamic range involves collecting two or more frames of the same scene to produce a single image. Each collected signal is stored in a separate frame buffer and processed at the end of the integration period. One variation of multiple integration is based on capturing two framesone with a long integration time followed by one shortand combining them. Information is captured from the darkest areas during the longer integration time and from the brightest areas during the shorter time. After a scaling factor is applied, the information from the separate integrations is merged to produce a wide dynamic range image Another variation of multiple integration is based on storing a preliminary image at a designated time and then continuing to collect charge until a nal image is stored the end of the integration period. The two images are compared, and if the charge at the

Charge

Noise Floor

Illumination

Figure 10. Ideal Response Curve

The ideal automotive camera and image sensor technology must also be: Built on reliable technology to ensure longlife operation Cost-effective to facilitate product introduction into the marketplace. Adaptive so that only the right amount of dynamic range expansion is applied to accommodate each scene. Programmable so that system designers can tailor the cameras response to specic conditions.

Table 2. Criteria for Ideal Automotive Wide Dynamic Range Cameras Criteria Linear response at low illumination Expanded high illumination response Reliable Cost-effective Adaptive Programmable Description Accumulated charge rises quickly above the noise oor. Curve is linear and steep-sloped at low illumination (high responsivity). Response is non-linear and shallow sloped at high illumination, expanding the capability to capture the bright light without reaching saturation. Uses proven technology and simple designs with minimal number of transistors per pixel. Requires minimal external processing, complex circuitry, or additional devices that could contribute to failure rate. Camera uses standard pixels and minimal external circuitry or devices that increase production expenses. Manufacturers can use one camera to perform multiple applications. Camera provides intelligent technology that adjusts its response curve to the conditions. Adaptive cameras improve contrast in low dynamic range scenes that dont require a highly non-linear response curve. Technology is exible and controllable. Manufacturers can program the response for specic conditions so that one camera model can be tailored to meet the needs of multiple applications.
6

WHITE PAPER
Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision
end of the integration period is saturated, the previously collected charge is used instead. Multiple integration can expand dynamic range. However, a major drawback is the external electronics required to implement this method. In addition to the image sensor, multiple integration requires an analog-todigital (A to D) conversion for each capture, a separate frame buffer to store each image, and external signal processing to perform operations on the multiple images. Table 3 summarizes the performance of multiple integration against the criteria for wide dynamic range cameras.
Logarithmic Response

related to the photocurrent. This mode of operation has signicantly higher xed pattern noise, making the low light illumination response considerably inferior to charge integration techniques. In addition, the frequency response of the pixel is dependent on the level of illumination. At low light level, the slow response causes image lag or ghosting. Table 4 summarizes the performance of logarithmic response against the criteria for wide dynamic range cameras.

combined technologies to expand dynamic range and a straightforward standard, threetransistor CMOS pixel to provide reliability and cost-effectiveness. No additional frame buffers are required. Based on research at the Massachusetts Institute of Technology, Autobrite controls the pixel through Melexis proprietary variable height/multiple reset method. A complete feedback loop simultaneously controls the integration time and dynamic range expansion for total adaptability and programmability. This section explains the variable height/multiple reset method and the feedback loop. This section also demonstrates how Autobrite meets all of the criteria for an ideal automotive camera.
Variable Height/Multiple Reset

6. The Autobrite Solution for Wide Dynamic Range Autobrite uses the variable height/ multiple reset method and a feedback loop to meet the criteria for an ideal automotive camera.

Some image sensors use a MOS transistor, operating below its threshold voltage to make use of a logarithmic dependence between the drain current and gate voltage. For this mode of operation, the response curve to achieve wide dynamic range is purely logarithmic. The output is no longer integrated charge, but a continuously changing voltage that is

Several methods of expanding dynamic range function by resetting the pixel at Autobrite is a complete solution from Melexis strategic times during the integration period to delay saturation by removing charge from that meets the photodiode. Autobrite takes this conall of the criteria proposed in Table 2 for wide dynamic range cameras. Autobrite uses cept further, using a variable height/mul-

Table 3. Performance Summary for Multiple Integration Criteria Low illumination response High illumination response Reliable Cost-effective Adaptive Programmable Evaluation Good. Expands dynamic range by applying a steep-sloped linear response curve, which raises the signal above the noise oor quickly. Good. Expands dynamic range by applying a shallow-sloped response curve, which delays the approach to saturation level. Poor. Requires additional frame buffers and external signal processing to store and process the images, increasing the number of electronic components that may fail. Poor. The cost of the external memory and complex circuitry required to store and process the images is a major drawback. In addition, multiple A/D conversions require faster converters and incrrease power consumption. Moderate Moderate

Table 4. Performance Summary for Logarithmic Response Criteria Low illumination response High illumination response Reliable Cost-effective Adaptive Programmable Evaluation Poor. logarithmic sensors have inherently more xed pattern noise than linear sensors. Image lag and ghosting are also apparent at low illumination. Good. Expands dynamic range by providing shallower-sloped response at high illumination. Untested. Logarithmic sensors have not been produced or eld utilized in high volumes. Untested. Logarithmic sensors have not been produced or eld utilized in high volumes. Poor. Not adaptive and thus unable to provide maximum contrast and data integrity in all scenes. Poor. Not programmable and thus limited in exibility.
7

WHITE PAPER
Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision
tiple reset to control not only the timing of the reset, but the height as well. To control the height of the reset, Autobrite uses a barrier voltage instead of a reset voltage, as shown in Figure 11.
V uc

M2

Photodiode

Photodiode and Barrier Voltage

P hotodiode M3 R ow S elect Output

VReset

Charge

B arrier

M1

If the voltage of the pixel is lower than V1 at the time of the pulse, as shown by the solid black line (representing high illumination), the voltage is reset to V1, essentially holding the pixel voltage at the barrier voltage and producing a nonlinear response. If the voltage of the pixel is higher than V1, as shown by the dotted black line (representing low illumination), the reset pulse does not affect the pixel voltage and the response is linear.

possible, ranging from linear to various degrees of non-linearity, some of which are shown in Figure 14. Multiple response curves provide exibility to select the best dynamic range for the given lighting conditions or application.

Illumination
V1

Figure 14 Multiple response curves Feedback Loop

Figure 11. Autobrite pixel architecture

Barrier Voltage

Barrier Voltage

Time

Before the integration period begins, the pixel voltage is driven near VUC, as with a standard active pixel. At the start of integration, the barrier voltage drops near zero and the photodiode voltage decreases with a slope that is dependent on the light intensity. During the integration process, the barrier voltage is increased to a programmable level. If the photocurrent reduces the pixel voltage to below the barrier voltage, M1 begins to conduct and holds the pixel voltage at the barrier voltage. This will only happen if the pixel voltage falls below the barrier voltage, indicating a high light intensity. If the pixel voltage remains above the barrier, it continues to integrate as a normal linear pixel. Figure 12 illustrates a simple variable height/ multiple reset example with two barriers or reset pulses. At the start of the integration period, reset pulse VReset is applied to the gate. During the integration period, a second reset pulse V1 is applied. The second pulse, which is lower in height than VReset, performs a conditional reset.

Figure 12. Variable height/multiple reset

The variable height/multiple reset method accommodates more complex arrangements of reset pulses. The spacing between the pulses and the height of the individual pulses are programmable and determine the response curve of the signal. Figure 13 shows a scenario with four reset pulses at diminishing barrier voltages VReset, V1, V2, and V3.
VReset V1 V2 V3

Achieving wide dynamic range through an image sensor with linear response at low illumination and non-linear response at high illumination solves only part of the problem. To complete the solution, the automotive camera must be able to decide which response curve to use. Furthermore, system designers must be able to override the automated decision to customize the response for specic applications. Autobrite uniquely meets these criteria with its key features: adaptability and programmability. Autobrite includes a mechanism to dynamically adjust both the response curve and the total integration time based on the scene being observed. Essentially, the dynamic range is expanded in real time by changing the timing and height of the reset signal. The control mechanism can be congured to automatically adapt to each environment or programmed for a specic application, thereby providing performance that is unmatched by other approaches for achieving wide dynamic range. Figure 15 illustrates the Autobrite control mechanism. The control loop starts with the image sensor capturing an image. Registers acquire statistics of the scene such as average

Voltage

Time

Figure 13. Multiple barrier operation

Variable height/multiple reset is capable of producing a linear response in low light as well as a non-linear response in bright light instantaneouslyall in the same image. In fact, a wide range of response curves are
8

WHITE PAPER
Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision
intensity and number of pixels that exceed a threshold. A proprietary control mechanism, which can be tailored to a specic application, uses the statistics to select the optimum response curve and integration time. A multiplexing device inputs the signals and allows either the calculated values or user-supplied values to be fed to the voltage and timing control, which generates the barrier voltage(s) for the image sensor. Simultaneously calculating both the integration time and the required dynamic range expansion allows the image sensor to settle on optimal settings very quickly. This fast response time is critical in applications where dramatic changes in the lighting conditions occur quickly, such as the appearance of headlights from other vehicles. Another advantage of this approach is that the collection of the statistics and the control algorithms can be tailored to a specic application. Original equipment manufacturers can program Autobrite to meet specic requirements of their application. For example, system engineers can program Autobrite to: Select a specic region of interest within the image frame that Autobrite will use to optimize the integration time and response curve. Select a specic integration time and response curve, overriding the automated adjustment. Select a maximum integration time or response curve that the automated adjustment is not to exceed. Adjust the speed of adaptibility to respond more quickly or slowly to lighting changes.
Autobrite Advantages

Figure 15. Autobrite feedback and control mechanism

Figure 16. Images captured without (left) and with (right) Autobrite Table 5 summarizes Autobrites ability to meet the criteria for wide dynamic range automotive cameras. Table 5. Performance Summary for Autobrite Low illumination response Good. At low illumination, Autobrite responds linearly, ensuring that the signal rises above the noise oor quickly. High illumination response Good. Expands dynamic range by modifying the response curve as needed by changing the height and timing of the barrier voltage pulses. Provides high illumination response varying from linear to highly non-linear. Good. Uses a standard, three-transistor CMOS pixel with no extra transistors or external storage. Using a standard pixel ensures access to proven and easily available technology as well as excellent position to incorporate advances in CMOS technology. SMaL image sensors incorporating Autobrite technology have been produced in volumes of millions Good. Uses the same pixel as linear image sensors with no extra cost associated with frame buffers. In addition to competitive production costs, no extra storage or A/D conversion means lower power consumption. Good. Autobrite includes a control mechanism to select the best response curve and integration time in real time to optimize dynamic range based on the intensity characteristics of the scene. Good. System engineers can apply control algorithms to customize Autobrite for specic conditions or applications.
9

Reliable

Figure 16 provides a side-by-side comparison of images captured with and without Autobrite. In the image on the left, the intrascene dynamic range clearly exceeds the dynamic range of the camera, resulting in clipping or lost detail in the light and dark regions. In the image on the right, Autobrite enables the same scene to be captured with complete visual details even in the extremes of brightness and darkness.

Cost-effective

Adaptive

Programmable

WHITE PAPER
Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision
Table 6 lists the methods described in this paper and summarizes their ability to meet the criteria for wide dynamic range.
Table 6 Summary of methods for expanding dynamic range Multiple Integration Linear response at low illumination Non-linear response at high illumination Reliable Cost-effective Adaptive Programmable Good Good Poor Poor Moderate Moderate Logarithmic Response Poor Good Untested Untested Poor Poor Autobrite Good Good Good Good Good Good

7. Autobrite in Automotive Applications Autobrite provides wide dynamic range as well as the ability to adapt to each environment or be programmed in real time.
Autobrite provides original equipment manufacturers with more options to optimize image capture for driver assistance applications. In addition to providing wide dynamic range, Autobrite has the ability to adapt to each environment or be programmed in real timeproviding machine vision manufacturers with unlimited exibility. In many cases, only one camera is needed for multiple applications. This section describes typical automotive applications for Autobrite-enabled cameras. These are only a few of the many examples of implementations in which Autobrite excels.
Lane Departure Warning

In the same tunnel scenario, the camera must adjust quickly to the lighting conditions as the driver exits the tunnel into the bright sunlight. Autobrite adapts the response curve and the integration time to the new scene, enabling the camera to continue capturing critical information about the road lane markings. Similarly, Autobrite automatically adapts between night and day lighting conditions, adjusting every frame to ensure total data integrity.
Vehicle Occupant Sensing

characteristics of the person. Autobrite can ignore the area next to or behind the person, eliminating extraneous information. In Figure 17, the image on the left is captured with a conventional camera that has adjusted the integration time for the bright area of the window, leaving the interior scene too dark. The image on the right, optimized with Autobrite for the most important physical region in the scene, resolves the interior occupant.
Driver Drowsiness

In applications where only a portion of the scene is of interest, system engineers can apply algorithms that ignore either physical or luminance regions when selecting the response curve. Autobrite optimizes only the designated area of the image, rather than the entire image. As an example, a vehicle occupant sensing application must determine whether or not a person is present and detect specic

In some situations, too much dynamic range can be detrimental. If a scene requires precise digital detail within a narrow range of illumination, an excessively wide dynamic range sensor would capture unnecessary information at the extremes of illumination and not enough information within the critical range.

A lane departure warning system must distinguish the road lane markings in all lighting and weather conditions in order to alert the driver when the vehicle moves out of its lane. Autobrites wide dynamic range enables the camera to capture critical visual information, detecting the lane markings even in a challenging scenario such as a dark tunnel with glare from the tunnel exit ahead.

Figure 17. Vehicle occupant sensing images captured with conventional (left) and Autobrite (right) technology
10

WHITE PAPER
Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision
For example, a driver drowsiness system that tracks eye movements might use a strobe ash to illuminate the subject. In this case, dynamic range is less important but high contrast is critical. Autobrites settings can be optimized to limit the dynamic range and provide maximum contrast. The automotive industry is rapidly developing new applications for electronically processing captured images. Autobrite-enabled cameras provide the wide dynamic range, adaptability, and programmability needed to optimize the front end of machine vision products. The effectiveness of any automotive machine vision application depends on a camera that can provide complete and accurate video information. As machine vision is increasingly applied to driver assistance and safety applications, manufacturers must develop systems that can process and respond to visual details in uncontrolled environments. The camera is the critical component that adapts to widely varying light levels. Wide dynamic range is the measure of a cameras ability to capture the difference between the brightest and the darkest details in a scene. Conventional cameras using image sensors with linear response do not provide the dynamic range needed for automotive applications. Alternative methods have been developed for expanding dynamic range, including multiple integration and logarithmic response. While these methods are successful in expanding dynamic range, each has signicant disadvantages. Autobrite proprietary technology from Melexis offers superior wide dynamic range performance to capture complete and accurate visual information for automotive machine vision systems. Autobrite is the only technology that offers adaptability and programmabilitythe ability to match the dynamic range of the image sensor to both the scene and the application. Figure 18 illustrates Autobrites superior wide dynamic range performance for a back-up assistance camera. In the image on the left, captured with a conventional camera, shadows obscure an object behind the vehicle because the camera has automatically adjusted the exposure to capture detail in the bright region. In the image on the right, Autobrite is able to capture both bright and dark details and clearly shows the obstacle in the cars path. Furthermore, Autobrite uses the same pixel structure as conventional linear sensors, providing the benets of competitive production costs, proven and easily available technology, and excellent position to incorporate advances in CMOS technology. In addition, no extra storage or A/D conversion means lower power consumption. Autobrite overcomes dynamic range limitations of machine vision in automotive applications. Autobrites adaptability and programmability ensure that complete and accurate visual details are captured even in the most challenging lighting conditions.

8. Conclusion Autobrite ensures that complete and accurate visual details are captured even in the most challenging lighting conditions.

Figure 18. Back-up assistance images captured with conventional (left) and Autobrite (right) technology

11

WHITE PAPER
Autobrite Imaging Technology: Wide Dynamic Range for Automotive Machine Vision
References 1. S. Decker, R.D. McGrath, K. Brehmer, and C.G. Sodini, A 256 x 256 CMOS Imaging Array with Wide Dynamic Range Pixels and Column-Parallel Digital Output, IEEE Journal of Solid-State Circuits, vol. SC-33, No. 12, pp. 2081-2091, Dec. 1998. 2. O. Yadid-Pecht and E.R. Fossum, Wide Intrascene Dynamic Range CMOS APS Using Dual Sampling, IEEE Transaction on Electron Devices, vol. ED-44, No. 10, pp 1721-1723, Oct. 1997. 3. D.X.D. Yang, A. El Gamal, B. Fowler, and H. Tian, A 640 x 512 CMOS Image Sensor with Ultrawide Dynamic Range Floating-Point Pixel-Level ADC, IEEE Journal of Solid-State Circuits, vol. SC-34, No. 12, pp. 1821-1834, Dec. 1999. 4. S.G. Chamberlain and J.P.Y. Lee, A Novel Wide Dynamic Range Silicon Photodetector and Linear Imaging Array, IEEE Journal of Solid-State Circuits, vol. SC-19, No. 2, pp. 41-48, Feb. 1984. 5. F.J. Kub and G.W. Anderson, Compressing Photodetectors for Long Optical Pulses Using a Lateral Blooming Drain Structure, IEEE Trans. Electron Devices, vol. ED-40, pp. 1740-1744, Oct. 1993. 6. S. Decker, A Wide Dynamic Range CMOS Imager with Parallel On-Chip Analog-to-Digital Conversion. PhD thesis, Massachusetts Institute of Technology, Sept. 1997. 7. K. Fife, A Stereo Vision System with Automatic Brightness Adaptation. Masters thesis, Massachusetts Institute of Technology, May 1995. 8. P. Acosta-Serani, A 256x256 Compact, High Resolution Imager for Machine Vision Applications. Masters thesis, Massachusetts Institute of Technology, Aug. 1997.
Autobrite is a registered registered trademark trademarkof ofSensata Melexis. Technologies. All names,trademarks trademarks and service marks are the property of their respective owners. All company company names, and service marks are the property of their respective owners.

Melexis, Inc Sensata Technologies

10 Wilson Road, 3rd Floor 529 Pleasant Street Cambridge, MA02703-2964 02138 Attleboro, MA U.S.A U.S.A. Phone 1-617-715-6668 1-508-236-3800 Phone www.sensata.com www.melexis.com
Copyright 2007, Sensata Technologies Copyright 2009, Melexis

Important Notice: reserves the right to makereserves changesthe to or discontinue any product or service identi ed Important Notice:Melexis Sensata Technologies (Sensata) right to make changes to or discontinue any product in this publication without notice. Melexis advises its customers to obtain latest version of the the latest relevant or service identied in this publication without notice. Sensata advises its the customers to obtain version of the information to verify, before placing any orders, that the information being relied uponrelied is current. assumes relevant information to verify, before placing any orders, that the information being upon Melexis is current. Sensata no responsibility for infringement of patents of orpatents rights of others on Melexis applications assistance or product assumes no responsibility for infringement or rightsbased of others based on Sensata applications assistance or speci cations since Melexis notdoes possess full access concerning the use oruse application of customers products. product specications since does Sensata not possess full access concerning the or application of customers prodMelexis also assumes no responsibility for customers product designs. ucts. Sensata also assumes no responsibility for customers product designs.

12

Printed in U.S.A, 2009 Printed in U.S.A., June, 2007