Vision sensors are becoming increasingly important edge devices for data acquisition. Originally used and developed as simple image sensors for photographic applications, today’s image sensors are used to deliver high-quality inputs feeding into AI and Machine Learning systems. These systems, in turn, have become sophisticated decision-making entities leveraging new and innovative processor architectures.
Edge Data Acquisition
Although edge data acquisition devices are predominantly analog in nature, image sensors are unique in that:
- Their outputs are time-multiplexed on a continuous, dynamic optical input
- They need the ability to uphold the integrity of the converted optical input while processing to deliver an image output
- The image output delivered is of optimal quality that allows for meaningful processing
These demands and the ensuing results could have a significant impact on the accuracy of the decisions made by vision systems, an outcome that defines the safety, reliability and profitability of the entire automated system.
Machine Learning-based Vision Systems
The advent of Machine Learning has driven image sensor innovation forward with enhanced levels of performance to support diverse applications. Visual input is high fidelity data – what you see is what is inputted into the system. Today, AI algorithms are capable of detecting, recognizing and classifying these inputs and generating accurate decision outputs. The dependability on these outputs is predicated on the quality of the inputs as much as they are on the accuracy of their algorithms, and the neural networks used for processing these algorithms
Vision Systems based on Machine Learning and Deep Learning predominantly use Convolutional Neural Network (CNN) algorithms to create a powerful automated recognition expert system. Increasing the depth of CNN layers in these systems increase inference accuracy, but more layers can also adversely affect the time it takes for these networks to learn in the training phase (not to forget the overfitting that could also result and the power consumed) and the latency in the system to complete an inference. Again, a high-quality image output enables the vision system to carry a minimal set of CNN layers, yet produce highly accurate inferences. This delivers significant benefits in terms of getting a quickly deployable intelligent system at low-cost and small form factor, yet delivering high performance and at low power consumption.
A Typical Convolutional Neural Network (CNN)
Deep learning algorithms such as CNNs are extremely resource intensive. A variety of processing engines are available today in the form of CPUs, GPUs, FPGAs, dedicated accelerators, and the recent microcontrollers. Designing a CNN-based vision system also requires a strong support of optimized libraries. This can range from proprietary (e.g., MVTec’s HALCON & MERLIC, MATLAB’s Deep Learning Toolbox, or Cognex’s ViDi) to standard tools (e.g., OpenCV) and integration capabilities of software & hardware. These choices have a direct bearing on the product’s time to market. Resource intensive processors generally demand larger form factors, power dissipation add-ons like heat sinks, or just large empty volume space to dissipate power through convection. Image sensors delivering high quality output can remove the need for expensive processors, costly 3rd party libraries and/or the creation of new ones, and costly tools needed to optimally tie the hardware and software resources. In other words, these sensors minimize the Total Cost of Ownership (TCO) significantly, and increase adoption in a wide variety of applications and markets.
Image Sensor Inputs to Machine Learning Systems
There are considerable demands on the image sensor output that is delivered to the CNN layers, including:
- A Global Shutter to be able to capture the scene and retain to minimize motion artifacts
- High global shutter efficiency to ensure the retained scene in every pixel is not corrupted by optical inputs that are outside of the light path of that pixel
- Sufficient pixel size to support good image quality even in challenging light conditions
- Low total noise in the image output to ensure a high-integrity input
- Low power during operation and standby to meet the typical challenges in a camera system where convection heat transfer is the norm.
These characteristics are dependent on the pixel architecture and on the design of the associated electrical path. The architecture and design of CMOS image sensors such as AR0234CS address these needs, making them ideal for CNN-based vision systems.
High-Speed Interface Enables Fast Systems
A pixel may be of excellent quality and perfectly designed to generate high-quality image, but the overall vision system might still suffer from poor performance due to bandwidth limitations. Sensors today are equipped with SerDes interfaces, but the rate at which these interfaces traffic data grades the overall system quality.
High frame rates require these interfaces to transfer image data at fast speeds. Also, sensors must consume low power for every frame output (fps/mW). These features allow system timing & power budgets to be transferred to where they are most desired – the processing engine - allowing state-of-the-art neural networks & complex algorithms to be incorporated. This enables the image processors to extract the nuances in the image data that could be material to the application. Vision System developers can thus differentiate their system solutions significantly against competition.
AR0234CS 2.3 Mp CMOS Image Sensor
With its high data rate MIPI interface, the AR0234CS 2.3 Mp CMOS image sensor is ideally suited for AI-based vision systems. Together with high frame rate, and low power consumption at full frame rate and full resolution, vision system developers can allocate a majority portion of their timing and power budgets to the processors.
Learn more about the AR0234CS or check out the below design resources.
Design Resources
Be sure to subscribe to our blog and follow us on social media to receive the latest updates on our technologies, solutions and company news!
Twitter | Facebook | LinkedIn | Instagram | YouTube
