Samsung is an innovator in the creation of high-performance image sensors that increase photo and video quality. Samsung recently held their Tech Day 2022 event, where they discussed the most recent advancements and future goals for numerous System LSI-related technologies, including the image sensor.
Joonseok Kim, Vice President and Head of the Project Management Team, explained the challenges of smartphone photography and videography, as well as how Samsung is developing image sensors to improve the quality of information shot with them.
Image sensor technology has advanced to the point that smartphone photos are now nearly similar to those shot with professional digital cameras. Sensor technology is critical, but software for multi-frame noise reduction, HDR, and AI, among other things, is also important.
Camera usage has also evolved significantly. It used to be that still photographs were mostly taken, but today it’s all about video. Video is deemed considerably more interesting, and with popular social media apps, customer demand for it is increasing. However, there is still a quality disparity between still photos and video, and reducing that gap has proven challenging.
Why Hasn’t Smartphone Video Caught Up?
The presence of visual “noise,” particularly in low light, is the primary reason smartphone video quality suffers. This is due to the insufficient exposure time required to catch sufficient light at 30 frames per second (fps) in video mode.
The second issue is that it is difficult to portray the real colours of both an item and the backdrop under complicated light circumstances such as backlighting with high-dynamic range (HDR) photos. To make this work, a smartphone must employ a system-on-chip (SoC) to do multi-exposure and multi-frame fusion, which requires a lot of memory and power.
The third limitation is a lack of depth-sensing capabilities. The “bokeh” effect, which emerges when part of an image is out of focus, is one of the greatest qualities of a DSLR camera. A user can utilise SoC to create images where the subject stands out from a blurred backdrop, but this approach isn’t viable for video for the same reasons that HDR is.
“We realised that we needed to handle these challenges using sensors rather than software, and we’re pursuing a three-part strategy. First, we’ll increase light and exposure sensitivity, which has been a major difficulty, particularly for small and thin smartphone cameras,” Kim explained. “Second, to improve the brightness of range we are working on 12-bit and 14-bit sensors, and towards even greater dynamic range sensors for outstanding HDR.
Finally, we are working on ToF (Time-of-Flight) sensors that detect genuine picture depth. “We want to create precise bokeh for smartphone video and other 3D applications.”
ISOCELL Advanced Pixel Technologies
Pixel creation science has advanced significantly in recent years. High-resolution image sensors, in general, need tiny, light-sensitive pixels. The pixel structure has developed from front-side illumination (FSI) to back-side illumination (BSI) in order to catch as much light as feasible (BSI).
The photodiode layer is now on top of the metal line rather than underneath it. Each pixel may catch more light by placing the photodiode closer to the light source. The disadvantage of this arrangement is that it increases crosstalk between pixels, resulting in colour pollution.
“To address this shortcoming, Samsung launched ISOCELL, its first technology that separates pixels from one another by adding barriers. ISOCELL is a combination of the terms “isolate” and “cell.” Kim expanded. “By isolating each pixel, ISOCELL may enhance the full well capacity of a pixel to store more light and eliminate crosstalk from one pixel to the next.”
ISOCELL pixel technology has grown consistently since its birth, and its current generation of innovations is being applied to the most recent image sensors. ISOCELL, for example, now has optical barriers built of novel low-refractive material between the colour filters. In addition to that, Kim revealed “We are creating another novel high refractive nano-structure to exploit the light of nearby pixels to extreme levels. We are getting high sensitivity that exceeds the conventional limitations by using nano-photonics technology.”
True-to-Life Colors with HDR Technologies
Conventional image sensors can only handle 10-bit pictures with a dynamic range of roughly 60 dB. However, in most applications, sensors must provide at least a 14-bit resolution to capture minute details. Multi-exposure picture capture is one method for still photographs. This approach uses a long-exposure photograph for the dark areas and a short-exposure image for the bright areas. The two photos are then combined to form an HDR image. Three or more exposures can be utilised to provide a greater dynamic range.
This approach has various drawbacks for videos. First, consider electricity usage. To get a 30 fps HDR movie, you’d have to film at 60 fps to get dual exposure. Processing such enormous amounts of picture data in real-time would need a massive amount of computing power.
The second issue is the difficulty of capturing moving things – motion artefacts. Using multiple exposures involves combining photographs obtained at various times. Any fast movement complicates the merging process and results in a distorted or strange picture.
“Instead, we are designing an in-sensor, single exposure HDR technology,” Kim said to assist overcome the issue. This allows the SoC process to be simplified, spend less power, and lessen the motion artefact impact for HDR video.”
A dynamic range is the ratio of the signal to the noise level. ISOCELL image sensors offer a very high full well capacity thanks to ISOCELL technology. The newest ISOCELL image sensor has pixels with up to 70,000 electrons, allowing the sensor to achieve a wide signal range.
“To decrease noise, we do two readouts: one with high gain to reveal the dark features and another with low gain to highlight the brilliant details,” Kim stated. In the sensor, the two readouts are then combined. Each read-out has ten bits. With the high conversion gain readout set to 4x, it adds an extra 2 bits, resulting in a 12-bit HDR image output. Smart-ISO Pro, commonly known as iDCG (intra-scene Dual Conversion Gain), is the name of this technology.
The ISOCELL HM6 and GN5 image sensors on the market today have Smart-ISO Pro and a 12-bit HDR function for crisp and detailed photos. However, 12-bit is still insufficient. To capture eye-catching video, at least a 14-bit resolution is required. Samsung just released the ISOCELL HP3, which has a 14-bit HDR with improved iDCG. Furthermore, Samsung intends to release 16-bit HDR sensors in the near future.
True Depth ToF Technology
For the past five years, Samsung has concentrated on ToF technology to determine the distance between a sensor and an object. ToF is the amount of time it takes for infrared light to bounce off an object and return to the sensor. In order to build a 2D depth map and 3D point cloud, Samsung’s indirect ToF (iTOF) delivers strong SNR, high resolution, and high frame rate.
Samsung intends to produce a new generation of iToF sensors with an integrated image signal processor (ISP). The whole-depth information processing is done on the ISP within the sensor rather than delegating to the SoC, resulting in decreased overall power usage.
Furthermore, the novel method improves depth quality even in low-light environments, thin objects, or repeated patterns. In the future, Samsung’s ISP-integrated ToF will aid in providing high-quality depth images with minimal to no motion blur or lagging at a high frame rate.
To get real-time news alerts join the Technewsrooms Telegram group. You can also follow us on Twitter and subscribe to our Google News feed for updates.
