Image processor
Animage processor,also known as animage processing engine,image processing unit(IPU), orimage signal processor(ISP), is a type ofmedia processoror specializeddigital signal processor(DSP) used forimage processing,indigital camerasor other devices.[1][2] Image processors often employparallel computingeven withSIMDorMIMDtechnologies to increase speed and efficiency. Thedigital imageprocessing engine can perform a range of tasks. To increase the system integration on embedded devices, often it is asystem on a chipwithmulti-core processorarchitecture.
Function[edit]
 |
Bayer transformation[edit]
Thephotodiodesemployed in animage sensorare color-blind by nature: they can only record shades of grey. To get color into the picture, they are covered with different color filters: red, green and blue (RGB) according to the pattern designated by theBayer filter- named after its inventor. As each photodiode records the color information for exactly onepixelof the image, without an image processor there would be a green pixel next to each red and blue pixel. (Actually, with most sensors there are two green for each blue and red diode.)
This process, however, is quite complex, and involves a number of different operations. Its quality depends largely on the effectiveness of the algorithms applied to the raw data coming from the sensor. The mathematically manipulated data becomes the photo file recorded.
Demosaicing[edit]
As stated above, the image processor evaluates the color and brightness data of a given pixel, compares them with the data from neighboring pixels, and then uses ademosaicingalgorithm to produce an appropriate colour and brightness value for the pixel. The image processor also assesses the whole picture to guess at the correct distribution of contrast. By adjusting thegammavalue (heightening or lowering the contrast range of an image's mid-tones), subtle tonal gradations, such as in human skin or the blue of the sky, become much more realistic.
Noise reduction[edit]
Noiseis a phenomenon found in any electronic circuitry. In digital photography its effect is often visible as random spots of obviously wrong colour in an otherwise smoothly-coloured area. Noise increases with temperature and exposure times. When higherISOsettings are chosen the electronic signal in the image sensor is amplified, which at the same time increases the noise level, leading to a lowersignal-to-noise ratio.The image processor attempts to separate the noise from the image information and to remove it. This can be quite a challenge, as the image may contain areas with fine textures which, if treated as noise, may lose some of their definition.
Image sharpening[edit]
As the color and brightness values for each pixel areinterpolatedsomeimage sharpeningis applied to even out any fuzziness that has occurred. To preserve the impression of depth, clarity and fine details, the image processor must sharpen edges and contours. It therefore mustdetect edgescorrectly and reproduce them smoothly and without over-sharpening.
Models[edit]
Image processor users are using industry standard products, application-specific standard products (ASSP) or evenapplication-specific integrated circuits(ASIC) with trade names: Canon's is calledDIGIC,Nikon'sExpeed,Olympus' TruePic, Panasonic'sVenus Engineand Sony'sBionz.Some are known to be based on theFujitsuMilbeaut,theTexas InstrumentsOMAP,PanasonicMN103,ZoranCoach, Altek Sunny orSanyoimage/video processors.
ARM architectureprocessors with itsNEON SIMDMedia Processing Engines(MPE) are often used inmobile phones.
Processor brand names[edit]
- ATI -Imageon(graphics co-processor used in many early mobile photos to offer camera image signal processing[3])
- Canon -DIGIC(based on Texas InstrumentsOMAP)[4]
- Casio - EXILIM engine
- Epson - EDiART
- Fujifilm - EXR III or X Processor Pro
- Google -Pixel Visual Core[5]
- HTC - ImageSense
- MediaTek - Imagiq
- Minolta / Konica Minolta -SUPHEEDwith CxProcess
- Leica - MAESTRO (based on FujitsuMilbeaut)[6]
- Nikon -Expeed(based on FujitsuMilbeaut)[7]
- Olympus - TruePic (based on PanasonicMN103/MN103S)
- OPPO - MariSilicon X
- Panasonic -Venus Engine(based on PanasonicMN103/MN103S)
- Pentax - PRIME (Pentax Real IMage Engine) (newer variants based on FujitsuMilbeaut)
- Qualcomm -Qualcomm Spectra
- Ricoh - GR engine (GR digital), Smooth Imaging Engine
- Samsung - DRIMe (based onSamsungExynos)
- Sanyo - Platinum engine
- Sigma - True
- Sharp - ProPix
- Socionext -MilbeautFamily of ISPs - SC2000 (M-10V), SC2002 (M-11S)
- Sony -Bionz
- THine - THP series[1]with compatible SDK Kit for developing firmware[2]
- UNISOC - Vivimagic
Speed[edit]
With the ever-higher pixel count in image sensors, the image processor's speed becomes more critical: photographers don't want to wait for the camera's image processor to complete its job before they can carry on shooting - they don't even want to notice some processing is going on inside the camera. Therefore, image processors must be optimised to cope with more data in the same or even a shorter period of time.
Software[edit]
libcamerais a software library that supports using image signal processors for the capture of pictures.
See also[edit]
References[edit]
- ^DIGITAL SIGNAL & IMAGE PROCESSING
- ^Fundamentals of digital image processing
- ^"Handheld Products".11 March 2006. Archived fromthe originalon 11 March 2006.Retrieved14 September2019.
- ^Inside the Canon Rebel T4i DSLRArchived2012-09-21 at theWayback MachineChipworks
- ^Amadeo, Ron (17 October 2017)."Surprise! The Pixel 2 is hiding a custom Google SoC for image processing".Ars Technica.Retrieved19 October2017.
- ^Fujitsu Microelectronics-Leica's Image Processing System Solution For High-End DSLRArchived2008-10-07 at theWayback Machine
- ^Milbeaut and EXPEEDArchived2016-05-21 at theWayback MachinebyThom