Acronym · Photonics, MEMS & Sensors
ISP (Image Signal Processor)
The fixed-function pipeline that turns raw sensor output into a viewable image, handling black level, defect correction, demosaic, white balance, noise reduction, tone mapping and sharpening.
- Also known as
- Image processor
- Type
- Acronym
- Domain
- Photonics, MEMS & Sensors, Image sensors
Notes
In phones the ISP increasingly hands off to neural accelerators, blurring the line with computational photography.
Overview
An ISP (image signal processor) is the dedicated processing pipeline that turns the raw output of an image sensor into a finished picture or video stream, correcting the sensor's imperfections and reconstructing full colour. It usually sits inside an application processor or camera SoC and receives sensor data over an interface such as MIPI CSI-2. Its design and tuning have a large effect on final image quality, alongside the sensor and lens.
How it works
Most sensors place a colour filter over each pixel in a Bayer pattern, so each pixel records only one colour. The ISP receives these raw values and runs them through a fixed sequence of stages. Early stages correct the sensor, typically subtracting the black level, fixing defective pixels and compensating for crosstalk and uneven lens shading. Demosaicing then estimates the two missing colours at every pixel from its neighbours. Later stages apply white balance and colour correction, reduce noise in space and over time, map the wide range of scene brightness into the output range, adjust gamma and sharpen edges. The result is usually a YUV stream for display, video encoding or computer vision.
Alongside the image, the ISP gathers statistics for the 3A functions: auto exposure, auto white balance and auto focus. Software, often called 3A libraries, reads these statistics each frame, calculates new settings and writes them back to the ISP and the sensor, closing a control loop that runs continuously while the camera is on.
ISPs differ in throughput, the number of cameras they can serve at once and the features they offer, such as combining several exposures into a high dynamic range image. Each new sensor and lens needs calibration and tuning of the ISP parameters. Most ISPs are largely fixed-function hardware configured by software, which suits the steady, high pixel rates involved.
Examples
- Arm's Mali-C55 ISP IP, which supports images up to 48 megapixels from up to eight sensors, with HDR stitching and temporal and spatial noise reduction.
- Intel's IPU3 imaging unit, documented in the Linux kernel, which takes raw Bayer frames and outputs processed video plus 3A statistics.
- The Rockchip rkisp1 ISP, whose Linux driver lets software change parameters such as black level while the camera is streaming.
- A driver assistance camera sending images over MIPI CSI-2 to the ISP in a vehicle's vision processor.
Why it matters
ISP hardware engineers design the pipeline, camera software engineers write drivers and 3A algorithms, image quality engineers tune it for each sensor, and verification engineers check it against reference models. In phones, cars, security cameras and drones, the ISP strongly influences how good images look and how well machine vision performs.
Related terms
Part of the Photonics, MEMS & Sensors glossary, with every term in the topic and the lifecycles, tools and standards that use them.
Where it appears on the Hub
Used on 13 pages of the Hub. In each group, the pages that mention it most come first.
Roles
Standards
Suggest an improvement
Suggest an improvement to this entry
A clearer definition, an example, a correction or a source an editor can check. Editors review every suggestion before the entry changes.
Last reviewed . Written by The Semiconductor Hub editorial team. The full glossary