Challenges and solutions for image acquisition and processing in portable designs

This four-part series of articles delves into the trends and design challenges of image acquisition and processing on mobile phones and other handheld device platforms. This section mainly discusses the use of software to enhance optical performance.

The invention of wafer-scale manufacturing technology makes it possible to produce its compact camera modules at very low cost. For physical reasons, the performance of a small camera module is inferior to that of a larger camera module, but the defects generated by the small camera module can be corrected by a new lens structure manufactured on a wafer scale. However, this can only be used to maintain the status quo, neither improve the image quality nor enhance the user experience.

Currently, designers of high-resolution camera phones can already highlight their pixel count to market their products to consumers. With the widespread use of camera phones (more than 80% of mobile phones have at least one camera installed), consumers have realized that there is no inevitable connection between the quality of the image and the number of pixels. In fact, the stunning pictures uploaded back from the rover were taken with a camera of only 1M pixels. Similarly, designers of professional-grade cameras have long known that in order to obtain a higher-quality digital picture, the close cooperation of optics and software is required.

As discussed in the first part of this article, customer needs have improved the image quality of a new camera phone and a range of embedded features. Among the most anticipated are optical zoom, focus adjustment, and low optical sensitivity, such as taking pictures without flash. All of these functions are relatively simple to implement, provided that the height and cost are allowed, especially the zoom range. Traditional zoom requires two lenses to perform relative movements on the optical axis in the camera. This can be done with a microdrive, but the final product will be bulky, power hungry, slow to respond, and not intact under the harsh environment of portable electronic devices, especially the "drop test." Therefore, how can the designers of camera models provide all the functions consumers need to improve the quality of pictures without affecting the appearance, reliability and most important cost of the product? The answer is software-enhanced optics.

Software Enhanced Optics

Software-enhanced optics, or "smart optics", is a correction technique for image processing of known optical effects. Optical effects can be inherent defects that must be corrected, or intentionally introducing an artificial correction that provides a function or a specific effect. If the goal is only to improve the quality of the image, you can buy high-quality, high-precision optical equipment without investment, and you can use software to correct the known distortion caused by the cheap optical lens magazine. For example, if the size and cost are limited, it means that there is always the same degree of blur in the corners of the picture. Software enhanced optics can apply edge sharpening algorithms to correct these corner areas. Then, users will be satisfied with such pictures, because the inherent defects in these camera modules can be corrected or masked, and the pictures presented are good in all departments. For further efficiency, this correction can be completely transparent to the user, that is, without user intervention.

With this software-enhanced optical concept, new opportunities are everywhere. These basic methods are to use professional lenses, operate these lights when they enter the camera, and distribute these lights on the imager according to the required functions. The operated picture is not used directly, it needs further correction by the software. However, because the image is manipulated in a known manner, it can be recovered digitally, so that high-quality output is presented. In this way, many functions can be achieved, including full optical zoom without moving parts, extended depth of field, and small F-value optical performance in low visibility environments.

Software-enhanced optical solutions for optical zoom take advantage of the phenomenon that in conventional optical lens magazines, the density of information is inconsistent across the field of view. The central area contains more data, while the edges are opposite. However, an image sensor has a regular, two-dimensional pixel array. This means that when framing, the center of the imager is normally sampled, and the edges are oversampled. The software-enhanced optical solution utilizes a specially designed pixel lens to provide non-uniformly distributed optical sensing performance within the field of view, which is in line with the quantization format of solid-state imagers. In effect, this is contrary to the traditional approach taken by nature. Many animals with single-aperture eyes, especially birds, have a standard "lens", but the ducts and cone cells on the retina are unevenly distributed. In both cases, the image will be distorted, but it can be corrected because the lens design and the pixel distribution of the imager (or retina) are known.

In order to be able to view at the same magnification, the algorithm must compress the details in the central area of ​​the field of view, because in this area the professional lens increases the magnification and resolution. Therefore, compression does not reduce the quality of the image, and in fact, the software-enhanced lens solution in such a design can make the image quality the same as that produced by traditional cameras. When the image is zoomed, the frame of the image will appear, and the center area that has been enlarged will be retained. Subsequently, the image will be corrected for distortion. This is the biggest difference from digital zoom, because zoom is the result of lens movement and has been fixed during image acquisition, so the zoomed image retains its original higher resolution. Software enhanced optics can achieve 3x zoom.

The figure is an example of using software to enhance the optics to provide the zoom function. In this solution, there are no moving parts, that is, it is physically compact, sturdy and durable, instantly visible, and consumes little power, and thus the cost is very low. This is significantly better than digital zoom. Digital zoom involves trimming and expanding the image to fill the field of view, which naturally reduces the resolution because the available information will be spread over a larger area. In a 3x digital zoom, almost 90% of the quality information will be lost during image acquisition, which is why digital zoom only provides a very small magnification function. The image enhancement solution for zooming can be implemented with a fixed lens and a simple algorithm. This makes this solution suitable for all imager technologies and all resolutions (from QCIF to> 10M pixels), so it is expected to be widely used in camera phones in the short term.

Figure. Use OPTIMLTM zoom software to enhance the optical zoom achieved by the optical solution (only the center area of ​​the field of view is displayed) (left) The image before distortion correction (middle) The image after distortion correction with 1x optical zoom magnification (right) Distorted image corrected by 2x optical zoom. Source: Tessera

The pictures obtained through the camera phone are often out of a "whim". Consumers do not want the scene to be arranged, and will not have the time or trouble to place the camera and themselves at the most appropriate distance from the target. With the help of these small optical devices, a traditional camera module only needs to focus on a target within a certain distance, typically 60 cm to tens of meters. Due to failure to understand and obey this limitation, consumers are often not satisfied with their own images. The image enhancement solution to this problem is "extended depth of field". This causes all the details of the scene to be fixed in focus, as long as the details are within a range of 10 cm to infinity from the camera module. Similar to the software-enhanced lens zoom solution, this can be obtained through the combination of optical magnification provided by professional lenses and a simple algorithm. It does not involve any moving parts, so it is rugged, reliable, instantaneous and low power consumption.

In the traditional camera module, the optical lens compartment is designed to focus the light source point, so it is placed at a certain distance from the imager in the camera. If the lens is out of focus or the target is too close to the lens, smudges will appear in the diffuse area, so the image will become blurred. This lens transforms the light source into a blurred light spot, which can be described as a mathematical change, called the point spread function. If the point spread function of the lens is known, the blur can be restored to the original scene by using digital signal processing. However, when only a certain area in the picture is out of focus, no matter what transformation method is used, it cannot be reliably recognized. Software-enhanced optics solves this problem by refocusing the entire image through a controllable method. The lens effectively creates an image with the same degree of blur regardless of the distance from the light source, which can be deinterleaved by a direct algorithm. As a result, the image is better and clearer, regardless of the foreground, middle distance, or distant view, all in focus. Figure 8 gives a good example.

Figure. A traditional lens can only focus on targets within a limited range, especially targets at medium and long distances; a software-enhanced optical solution, such as OpTIML focusing, can achieve extended depth of field from 10 cm to infinity without the need for Increase the height or complexity of the camera module. Source: Tessera

One of the main complaints about camera phones is their performance at low visibility. In fact, this is just a semi-truth proposition. The reduction in pixel size of small camera modules has undoubtedly led to a reduction in optical sensitivity relative to digital still cameras. From a pixel size of 2.2 μm in 2007 to 1.75 μm in 2008, it is expected to grow to 1.4 μm in 2009 and eventually reach 1.1 μm. This trend will have a significant impact on low visibility performance and image quality. In short, as the pixel size decreases, so does its sensitivity. From a more technical point of view, the ability of a photodiode to absorb photons and release electrons decreases as pixels decrease. Other corresponding effects brought by the small pixel size include the first dynamic range and the reduced signal-to-noise ratio. In reality, the poor perception of camera phones with low visibility performance is mainly caused by more and more taking pictures in low visibility environments; typically, at night and in places such as clubs and restaurants, where The light intensity is about 5 lux, which is far less than 350 lux outdoors during the day. Due to the decrease in brightness, the quality of the pictures obtained from the digital imager naturally deteriorates rapidly, and defects, loss of details, or color errors appear as if the noise were increased.

An important reason for the insufficient visibility performance of camera phones is that the F value of the optical lens compartment cannot be changed because it must be fixed at the time of manufacture. Most digital still cameras provide an option to increase the aperture to compensate for the reduction in the number of photons in dim scenes. However, the mechanically adjustable iris will make the fuselage larger, less robust, slower in response, and higher in power consumption. Simply lowering the F value of the fixed aperture camera to improve low light sensitivity is not a good choice, because a larger aperture will reduce the depth of field, making it difficult to obtain a good image quality when the scene has a depth of field. Typically, the aperture of a standard camera phone is between F / 2.8 and F / 2.4, mainly to preserve enough depth to focus under normal brightness conditions. A simple imaging aperture is fixed, and the exposure time cannot be extended under low light conditions. However, this makes the picture susceptible to motion blur or body shaking, and is not possible for video capture, because video capture requires exposure time limited by the frame rate to within 67 milliseconds.

"Speed" is a simple way to describe the ability of an optical system to pass light to an imager. Operate the "slow motion" in a well-lit environment. This is because optics allows the use of a small aperture and a slow shutter to obtain a good depth of field. When taking pictures in poor light or in good light but requires a fast shutter speed (such as the sports mentioned below), a "fast shot" is required. Therefore, the challenge is to provide a good connection between lighting conditions, depth of field, and shutter speed, and to develop a fast lens for low-light scenes.

Software-enhanced optics provides a fully automated solution for camera phones, which allows consumers to take clear pictures under a wider range of lighting conditions. The basic idea of ​​this method is to design a camera module with low-F optics, typically F / 1.75, and restore the depth of field to a normal condition through one of the above-mentioned extended depth of field solutions. Low-F optics make ultra-high-speed lens solutions suitable for still photography and video capture. Signal processing can compensate for the loss of contrast and then reduce the noise in the final image, while retaining the edges, details, and texture of the original image. This is possible because the information written to the linear buffer for use by the algorithm provides pixel-based data and improves the signal-to-noise ratio of the image by about 6 dB. The effectiveness of this scheme can be described by comparing the two images obtained by the 1.75 μm imager in FIG. 9.

achieve

Software-enhanced optics combines professional lenses and custom algorithms to provide pictures with excellent quality and is completely transparent to users. However, designers of camera modules need to consider in advance how to incorporate these enhancements into the handset, rather than a plug-in form. In principle, all the requirements are just a lens in the optical lens warehouse developed by the customer, which can be manufactured through the existing architecture and lens materials. Custom lenses can even replace existing lenses. Along with this lens, there are image processing algorithms. The algorithms used in these schemes can be very small, usually 100,000 logic gates. This is very small for embedding in a CMOS imager, but this requires coordination with the imager manufacturer, and then the mold must be placed with the correct optics.

Another method of placing the algorithm is in the form of software or firmware, which can run on the image processor or mobile phone processor. Similarly, both solutions are technically simple, but need to communicate well with traditional camera module suppliers. However, the benefits of these solutions are so compelling, a 3-megapixel camera phone with extended depth of field is already in mass production and will be equipped with high-resolution cameras in 2009-together with zoom and ultra-fast lens solutions.

Although these software enhancements that enhance the original performance of highly miniaturized and low-cost camera modules are independent of zoom solutions, they do not provide functions that can increase user satisfaction during the shooting experience. This issue does not involve camera module designers, but the task of original equipment manufacturers. The most common problem with a digital camera is the red-eye phenomenon, which explains why more than 80% of current digital still cameras implement the function of reducing red-eye phenomenon. Whether these features will be provided on camera phones, and how they are integrated, we will discuss in the fourth part of this series of articles.

The FirstPower High Rate Discharge Battery is desgined for apllcation as High Rate Discharge. This series is a high energy, high intensity, and high quality output electrical energy product series with 20% more energy density than same volume products at high rate discharge characteristics.

We welcome orders with "FirstPower" brand; We are also flexible to accept orders on OEM basis. Contact us now! Your partnership with FirstPower will prove worthy of it.

High Rate Discharge Battery

High Rate Discharge Battery,High Discharge Rate Lipo Battery,Smart High Rate Discharge Battery

Firstpower Tech. Co., Ltd. , https://www.firstpowersales.com