So you want to learn about ISP & camera tuning?
Video images that ultimately make it to your display or are used for computer vision are typically first captured by a lens and sensor. These raw images could have noise from the environment, unwanted artifacts, poor color, limited lighting, and many other issues that impact the image quality. An Image Signal Processor (ISP) is used to perform operations on the video images and convert them to digital streams that are easier to handle for downstream systems.
ISPs often have many settings that help enable features like sharpness, gamma correction, auto exposure, white balance, black level, and gain adjustments. ISPs can have hundreds to thousands of different parameters to adjust. Setting up these parameters to match the required application requires extensive manual tuning.
Tuning involves testing the camera and ISP with different settings to achieve the best image quality. Due to the high number of parameters, this tuning can take several months of trial and error intuition to achieve an optimized tuning. Also, if the camera is used for computer vision (helping machines see the environment) then it is extremely difficult for a person to determine what settings produce the optimal images for these computer algorithms to analyze.
Atlas Camera Optimization Suite Overview
The Atlas Camera Optimization Suite is the industry’s first set of machine-learning tools and workflows that automatically optimizes camera architectures for computer vision. Atlas significantly improves computer vision results in days vs. traditional approaches that deliver suboptimal results even after many months of manual tuning.
Simply put, it automates the painful manual task of tuning cameras and ISPs. The Atlas tuning methodology uses metrics, also known as Key Performance Indicators (KPIs) to achieve certain objectives. The solvers then adjust the ISP parameters over thousands of iterations to achieve the KPI targets.
This approach minimizes the use of subjective and manual adjustments and has been shown to produce much better results. Atlas tuned image quality for visual applications, like a vehicle rearview camera, can have improved color, cleaner edges, and fewer unwanted artifacts. Image quality tuned by Atlas for computer vision also has been shown to produce characteristics that improve accuracy when identifying and tracking objects.
Atlas can iterate thousands of settings over days that could take a person many months to achieve a similar result. The Atlas application and methodology can prevent time and resources wasted on trial and error. It can also achieve better quality for visual and computer vision applications.
Atlas Set Up
The Atlas methodology and application requires setting up a camera and ISP within a testing environment. Below is a typical setup for Atlas for tuning visual image quality:
Atlas setup for visual image quality tuning includes a number of components within your lab environment:
- Display Machine with 4K Display- Mini PC
- ISP board with camera
- Control machine – runs camera system and connects to camera hardware
- Compute machine- mini PC or Notebook that runs Atlas
- Additional charts and lighting is required for HDR and Auto White Balance tuning
The ISP board and camera are set up and oriented towards the 4K display. The 4K display is powered by a mini PC that shows a multi-objective tuning chart. A control PC is used to run the camera system and connect to the camera hardware. Finally, a compute machine is used to run the Atlas software and determine the optimized parameters.
Setup for Atlas CV is actually simpler and only needs computer vision KPIs to be calculated by the computer vision algorithm, such as mean Average Precision (mAP) for object detection. It does not require the setup of an LCD screen or camera, but only a small dataset of annotated raw images that is representative of the typical use cases and the ability to directly inject those images repeatedly.
Atlas Workflow Methodology
After the Atlas testing environment is set up, the Atlas workflow can be used to tune the ISP and camera sensor. If the ISP is new, there is an integration step required to integrate it into Atlas. But, many ISPs are already currently integrated with Atlas, shortening the setup process.
The lab and KPI targets are then set to achieve the desired objectives. The solvers can then be applied to achieve the KPI targets. Results can be analyzed and optimal parameters identified while the optimization is running or after the iterations are complete. The workflow is summarized here.
- Lab environment setup
- ISP integration (if needed)
- Set KPI targets
- Solver optimizes by iterating towards the KPI targets
- Results analysis
- Optimal ISP settings
When the optimal ISP settings are found they can be used for the camera and ISP combination. If the camera use case changes or if a new camera sensor is needed then the tuning set up can be re-used to optimize the settings again.