Much has been written about the rapid pace at which automated driver assistance systems (ADAS) are proliferating in modern vehicles. These systems are designed with a primary goal of improving road safety and are steps on the path to the fully autonomous vehicles of the future.
Increasingly, vehicles have multiple cameras looking externally at the surroundings of the vehicle to detect and identify objects in the path of the vehicle, or to read roadside signs and control the vehicle. Other cameras monitor the periphery of the vehicle, keeping it in a lane on the highway or helping to park safely in crowded spaces.
While great strides to these life-saving systems are being made, the most dangerous aspect of any vehicle remains the driver and, more specifically, their lack of attention and poor decision-making. Sometimes referred to as ‘the nut behind the wheel’, the driver has been shown by NHTSA surveys to be responsible for in excess of 90% of all road traffic accidents.
Automakers and safety regulators are recognizing that ADAS systems can only go so far and, if the driver is not paying attention, then the chances of an accident remain extremely high – even with ADAS in place. As a result, cameras and vision systems are now being added internally to monitor the driver and, increasingly, the other occupants as well.
In fact, these driver monitoring cameras (DMC) are considered to be so important to road safety, Euro NCAP has made them a primary safety feature from 2020. Through discreetly aiming a camera at the driver and adding advanced image processing software, the ADAS system can determine a number of things that will contribute to road safety.
DMC-equipped ADAS systems can tell if the driver is looking at the road ahead, or trying to read a message on their smartphone. It can detect if the driver has been taken seriously ill and become incapacitated and more advanced systems are able to deduce the emotional state of the driver using facial expressions as cues.
In the event that an accident should happen, the same camera can detect the size, weight and position of the driver and adjust the airbag deployment accordingly, reducing the risk of injury.
Expanding the view are occupant monitoring cameras (OMC) that further contribute to safety by detecting the presence of other people in the cabin. With OMC, ADAS systems are able to determine if children or pets are present and whether seatbelts are being used. One such future mandate is the detection of the presence of children in the vehicle, specifically children left alone unattended.
While safety is the primary goal of DMC and OMC, they also provide a number of convenience features. DMC are able to detect the driver and adjust the cabin (seat position, climate control etc.) to their preferences. Advanced biometrics can be used to authorize payments when using toll roads. OMC can record dangerous passengers in taxis, providing a deterrent to keep the driver safe.
To demonstrate the capabilities of this technology, ON Semiconductor has collaborated with ecosystem partners and developed a complete in-cabin monitoring system that includes DMC and OMC. At the heart of the system is a 1 Megapixel (MP) global shutter AR0144AT image sensor along with three 2.3 MP RGB-IR image sensors. The multi-camera system utilizes Ambarella’s CV2AQ system-on-chip (SoC) that processes real-time high-quality RGB-IR video and integrates Eyeris’ AI software performing complex body and facial analytics, passenger activity monitoring, and object detection.
Driver and occupant monitoring applications require the ability to capture images in variable lighting from direct sunlight to pitch black conditions. With outstanding NIR response, the RGB-IR CMOS image sensor technology provides full HD 1080p output using a 3.0 µm backside illuminated (BSI) and three-exposure HDR. Sensitive to both RGB and IR light, the sensors are able to capture color images in daylight and monochrome IR images with NIR illumination.
2
