Wikipedia

Chroma subsampling

Chroma subsamplingis the practice of encoding images by implementing less resolution forchromainformationthan forlumainformation, taking advantage of the human visual system's lower acuity for color differences than for luminance.[1]

Widely used chroma subsampling formats

It is used in many video and still image encoding schemes – both analog and digital – including inJPEGencoding.

Rationale

edit
Infull size,this image shows the difference between four subsampling schemes. Note how similar the color images appear. The lower row shows the resolution of the color information.

Digital signals are often compressed to reduce file size and save transmission time. Since the human visual system is much more sensitive to variations in brightness than color, a video system can be optimized by devoting more bandwidth to thelumacomponent (usually denoted Y'), than to the color difference componentsCbandCr.In compressed images, for example, the 4:2:2Y'CbCrscheme requires two-thirds the bandwidth of non-subsampled "4:4:4"R'G'B'.[a]This reduction results in almost no visual difference as perceived by the viewer.

How subsampling works

edit

Thehuman vision system(HVS) processes color information (hueandcolorfulness) at about a third of the resolution ofluminance(lightness/darkness information in an image). Therefore it is possible tosamplecolor information at a lower resolution while maintaining good image quality.

This is achieved by encodingRGBimage data into a compositeblack and whiteimage, with separated color difference data (chroma). For example with,gamma encodedcomponents are weighted and then summed together to create thelumacomponent. The color difference components are created by subtracting two of the weightedcomponents from the third. A variety offilteringmethods can be used to limit the resolution.

Regarding gamma and transfer functions

edit

Gamma encoded lumashould not be confused with linearluminance.The presence of gamma encoding is denoted with theprime symbol.

Gamma-correcting electro-optical transfer functions (EOTF) are used due to the nonlinear response of human vision. The use of gamma improves perceived signal-to-noise in analogue systems, and allows for more efficient data encoding in digital systems. This encoding uses more levels for darker colors than for lighter ones, accommodating human vision sensitivity.[2]

Sampling systems and ratios

edit

The subsampling scheme is commonly expressed as a three-part ratioJ:a:b(e.g. 4:2:2) or four parts, if alpha channel is present (e.g. 4:2:2:4), that describe the number of luminance and chrominance samples in a conceptual region that isJpixels wide and 2 pixels high. The parts are (in their respective order):

  • J:horizontal sampling reference (width of the conceptual region). Usually, 4.
  • a:number of chrominance samples (Cr,Cb) in the first row ofJpixels.
  • b:number of changes of chrominance samples (Cr,Cb) between first and second row ofJpixels.bis usually either zero or equal toa(except in rare irregular cases like 4:4:1 and 4:2:1, which do not follow this convention).
  • Alpha:horizontal factor (relative to first digit). May be omitted if alpha component is not present, and is equal toJwhen present.

This notation is not valid for all combinations and has exceptions, e.g. 4:1:0 (where the height of the region is not 2 pixels, but 4 pixels, so if 8 bits per component are used, the media would be 9 bits per pixel) and 4:2:1.

4:1:1 4:2:0 4:2:2 4:4:0 4:4:4
Y'CrCb
= = = = =
Y'
+ + + + +
1 2 3 4 J = 4 1 2 3 4 J = 4 1 2 3 4 J = 4 1 2 3 4 J = 4 1 2 3 4 J = 4
(Cr, Cb) 1 a =1 1 2 a =2 1 2 a =2 1 2 3 4 a =4 1 2 3 4 a =4
1 b =1 b =0 1 2 b =2 b =0 1 2 3 4 b =4
¼ horizontal resolution,
full vertical resolution 		½ horizontal resolution,
½ vertical resolution 		½ horizontal resolution,
full vertical resolution 		full horizontal resolution,
½ vertical resolution 		full horizontal resolution,
full vertical resolution

The mapping examples given are only theoretical and for illustration. Also the diagram does not indicate any chroma filtering, which should be applied to avoidaliasing.To calculate required bandwidth factor relative to 4:4:4 (or 4:4:4:4), one needs to sum all the factors and divide the result by 12 (or 16, if alpha is present).

Types of sampling and subsampling

edit

4:4:4

edit

Each of the threeY'CbCrcomponents has the same sample rate, thus there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and cinematic post-production.

"4:4:4" may instead be wrongly referring toR'G'B'color space, which implicitly also does not have any chroma subsampling (except in JPEG R'G'B' can be subsampled). Formats such asHDCAM SRcan record 4:4:4 R'G'B' over dual-linkHD-SDI.

4:2:2

edit

The two chroma components are sampled at half the horizontal sample rate of luma: the horizontal chroma resolution is halved. This reduces the bandwidth of an uncompressed video signal by one-third, which means for 8 bit per component withoutalpha(24 bit per pixel) only 16 bits are enough, as in NV16.

Many high-end digital video formats and interfaces use this scheme:

4:1:1

edit

In 4:1:1 chroma subsampling, the horizontal color resolution is quartered, and the bandwidth is halved compared to no chroma subsampling. Initially, 4:1:1 chroma subsampling of theDVformat was not considered to be broadcast quality and was only acceptable for low-end and consumer applications.[3][4]However,DV-based formats (some of which use 4:1:1 chroma subsampling) have been used professionally in electronic news gathering and in playout servers. DV has also been sporadically used in feature films and indigital cinematography.

In the480i"NTSC" system, if the luma is sampled at 13.5 MHz, then this means that theCrandCbsignals will each be sampled at 3.375 MHz, which corresponds to a maximumNyquistbandwidth of 1.6875 MHz, whereas traditional "high-end broadcastanalog NTSCencoder "would have a Nyquist bandwidth of 1.5 MHz and 0.5 MHz for theI/Qchannels. However, in most equipment, especially cheap TV sets andVHS/BetamaxVCRs,the chroma channels have only the 0.5 MHz bandwidth for bothCrandCb(or equivalently for I/Q). Thus the DV system actually provides a superior color bandwidth compared to the bestcomposite analogspecifications for NTSC, despite having only 1/4 of the chroma bandwidth of a "full" digital signal.

Formats that use 4:1:1 chroma subsampling include:

4:2:0

edit

In 4:2:0, the horizontal sampling is doubled compared to 4:1:1, but as theCbandCrchannels are only sampled on each alternate line in this scheme, the vertical resolution is halved. The data rate is thus the same. This fits reasonably well with thePALcolor encoding system, since this has only half the vertical chrominance resolution ofNTSC.It would also fit extremely well with theSECAMcolor encoding system, since like that format, 4:2:0 only stores and transmits one color channel per line (the other channel being recovered from the previous line). However, little equipment has actually been produced that outputs a SECAM analogue video signal. In general, SECAM territories either have to use a PAL-capable display or atranscoderto convert the PAL signal to SECAM for display.

Different variants of 4:2:0 chroma configurations are found in:

CbandCrare each subsampled at a factor of 2 both horizontally and vertically. Most digital video formats corresponding to 576i "PAL" use 4:2:0 chroma subsampling.

Sampling positions

edit

There are four main variants of 4:2:0 schemes, having different horizontal and vertical sampling siting relative to the 2×2 "square" of the original input size.[15]

  • In MPEG-2, MPEG-4, and AVC,CbandCrare taken on midpoint of the left-edge of the 2×2 square. In other words, they have the same horizontal location as the top-left pixel, but is shifted one-half pixel down vertically. Also called "left".[16]
  • In JPEG/JFIF, H.261, and MPEG-1,CbandCrare taken at the center of 2×2 the square. In other words, they are offset one-half pixel to the right and one-half pixel down compared to the top-left pixel. Also called "center".[16]
  • In HEVC for BT.2020 andBT.2100content (in particular on Blu-rays),CbandCrare sampled at the same location as the group's top-left Y pixel ( "co-sited", "co-located" ). Also called "top-left". An analogous co-sited sampling is used in MPEG-2 4:2:2.[16]
  • In 4:2:0 PAL-DV (IEC 61834-2),Cbis sampled at the same location as the group's top-left Y pixel, butCris sampled one pixel down.[17]It isalsocalled "top-left" in ffmpeg.[16]
Interlaced and progressive
edit

Withinterlacedmaterial, 4:2:0 chroma subsampling can result in motion artifacts if it is implemented the same way as for progressive material. The luma samples are derived from separate time intervals, while the chroma samples would be derived from both time intervals. It is this difference that can result in motion artifacts. The MPEG-2 standard allows for an alternate interlaced sampling scheme, where 4:2:0 is applied to each field (not both fields at once). This solves the problem of motion artifacts, reduces the vertical chroma resolution by half, and can introduce comb-like artifacts in the image.


Original. This image shows a single field. The moving text has some motion blur applied to it.


4:2:0progressivesampling applied to movinginterlacedmaterial. The chroma leads and trails the moving text. This image shows a single field.


4:2:0interlacedsampling applied to movinginterlacedmaterial. This image shows a single field.

In the 4:2:0 interlaced scheme, however, vertical resolution of the chroma is roughly halved, since the chroma samples effectively describe an area 2 samples wide by 4 samples tall instead of 2×2. As well, the spatial displacement between both fields can result in the appearance of comb-like chroma artifacts.


Original still image.


4:2:0progressivesampling applied to a still image. Both fields are shown.


4:2:0interlacedsampling applied to a still image. Both fields are shown.

If the interlaced material is to be de-interlaced, the comb-like chroma artifacts (from 4:2:0 interlaced sampling) can be removed by blurring the chroma vertically.[18]

4:1:0

edit

This ratio is possible, and somecodecssupport it, but it is not widely used. This ratio uses half of the vertical and one-fourth the horizontal color resolutions, with only one-eighth of the bandwidth of the maximum color resolutions used. Uncompressed video in this format with 8-bit quantization uses 10 bytes for every macropixel (which is 4×2 pixels) or 10 bit for every pixel. It has the equivalent chrominance bandwidth of a PAL-I or PAL-M signal decoded with a delay line decoder, and still very much superior to NTSC.

3:1:1

edit

Used by Sony in their HDCAM High Definition recorders (not HDCAM SR). In the horizontal dimension, luma is sampled horizontally at three quarters of the full HD sampling rate – 1440 samples per row instead of 1920. Chroma is sampled at 480 samples per row, a third of the luma sampling rate. In the vertical dimension, both luma and chroma are sampled at the full HD sampling rate (1080 samples vertically).

Different Cb and Cr rates

edit

A number of legacy schemes allow different subsampling factors in Cb and Cr, similar to how a different amount of bandwidth is allocated to the two chroma values in broadcast systems such asCCIR System M.These schemes are not expressible inJ:a:bnotation. Instead, they adopt aY:Cb:Crnotation, with each part describing the amount of resolution for the corresponding component. It is unspecified whether the resolution reduction happens in the horizontal or vertical direction.

  • In JPEG, 4:4:2 and 4:2:1 half the vertical resolution ofCbcompared to 4:4:4 and 4:4:0.[19]
  • In another version of4:2:1,Cbhorizontal resolution is half that ofCr(and a quarter of the horizontal resolution ofY).
  • 4:1:0.5 or 4:1:0.25 are variants of 4:1:0 with reduced horizontal resolution on Cb, similar to VHS quality.

Artifacts

edit
Original image without color subsampling. 200% zoom.
Image after color subsampling (Sony Vegas DV codec, box filtering.)
Note the bleeding in lightness near the borders.

Chroma subsampling suffers from two main types of artifacts, causing degradation more noticeable than intended where colors change abruptly.

Gamma luminance error

edit
Further information:Gamma correction § Scaling and blending

Gamma-corrected signals like Y'CbCr have an issue where chroma errors "bleed" into luma. In those signals, a low chroma actually makes a color appear less bright than one with equivalent luma. As a result, when a saturated color blends with an unsaturated or complementary color, a loss of luminance occurs at the border. This can be seen in the example between magenta and green.[20]This issue persists in HDR video where gamma is generalized into a transfer function "EOTF".A steeper EOTF shows a stronger luminance loss.[21]

Some proposed corrections of this issue are:

  • Luma-weighted average (Kornelski, experiment for mozjpeg)[22]
  • Iterativesharp YUVmethod, used byWebPand optionallyAVIF.Sharp YUV assumes abilinearupscaling for chroma.[23]
  • RGB subsampling in linear space before chroma subsampling (HDRTools)[21]
  • Iterative or closed-form luma correction to minimize luminance error (HDRTools)[24]

Rec. 2020defines a "constant luminance" Yc'CbcCrc, which is calculated from linear RGB components and then gamma-encoded. This version does not suffer from the luminance loss by design.[25]

Gamut clipping

edit

Another artifact that can occur with chroma subsampling is that out-of-gamutcolors can occur upon chroma reconstruction. Suppose the image consisted of alternating 1-pixel red and black lines and the subsampling omitted the chroma for the black pixels. Chroma from the red pixels will be reconstructed onto the black pixels, causing the new pixels to have positive red andnegativegreen and blue values. As displays cannot output negative light (negative light does not exist), these negative values will effectively be clipped, and the resulting luma value will be too high. Other sub-sampling filters (especially the averaging "box" ) have a similar issue that is harder to make a simple example out of. Similar artifacts arise in the less artificial example of gradation near a fairly sharp red/black boundary.[20]

It is possible for the decoder to deal with out-of-gamut colors by considering how much chroma a given luma value can hold and distribute it into the 4:4:4 intermediate accordingly, termed "in-range chroma reconstruction" by Glenn Chan. The "proportion" method is in spirit similar to Kornelski's luma-weighted average, while the "spill" method resembleserror diffusion.[20]Improving chroma reconstruction remains an active field of research.[26]

Terminology

edit

The termY'UVrefers to an analog TV encoding scheme (ITU-R Rec. BT.470) while Y'CbCr refers to a digital encoding scheme.[2]One difference between the two is that the scale factors on the chroma components (U, V, Cb, and Cr) are different. However, the term YUV is often used erroneously to refer to Y'CbCr encoding. Hence, expressions like "4:2:2 YUV" always refer to 4:2:2 Y'CbCr, since there simply is no such thing as 4:x:x in analog encoding (such as YUV). Pixel formats used in Y'CbCr can be referred to as YUV too, for example yuv420p, yuvj420p and many others.

In a similar vein, the term luminance and the symbol Y are often used erroneously to refer to luma, which is denoted with the symbol Y'. Theluma(Y') of video engineering deviates from theluminance(Y) of color science (as defined byCIE). Luma is formed as the weighted sum ofgamma-corrected(tristimulus) RGB components. Luminance is formed as a weighed sum oflinear(tristimulus) RGB components. In practice, theCIEsymbol Y is often incorrectly used to denote luma. In 1993,SMPTEadopted Engineering Guideline EG 28, clarifying the two terms. The prime symbol ' is used to indicate gamma correction.[27]

Similarly, the chroma of video engineering differs from the chrominance of color science. The chroma of video engineering is formed from weighted tristimulus components (gamma corrected, OETF), not linear components. In video engineering practice, the termschroma,chrominance,andsaturationare often used interchangeably to refer to chroma, but it is not a good practice, as ITU-T Rec H.273 says.[28]

History

edit

Chroma subsampling was developed in the 1950s byAlda Bedfordfor the development of color television byRCA,which developed into theNTSCstandard; luma–chroma separation was developed earlier, in 1938 byGeorges Valensi.Through studies[which?],he showed that the human eye has high resolution only for black and white, somewhat less for "mid-range" colors like yellows and greens, and much less for colors on the end of the spectrum, reds and blues.[clarification needed]This knowledge allowed RCA to develop a system in which they discarded most of the blue signal after it comes from the camera, keeping most of the green and only some of the red; this is chroma subsampling in theYIQcolor space and is roughly analogous to 4:2:1 subsampling, in that it has decreasing resolution for luma, yellow/green, and red/blue.

See also

edit

Notes

edit
  1. ^The prime signs indicates gamma-correction or any non-linear EOTF.

References

edit
  1. ^ S. Winkler, C. J. van den Branden Lambrecht, and M. Kunt (2001)."Vision and Video: Models and Applications".In Christian J. van den Branden Lambrecht (ed.).Vision models and applications to image and video processing.Springer. p. 209.ISBN978-0-7923-7422-0.{{cite book}}:CS1 maint: multiple names: authors list (link)
  2. ^abPoynton, Charles."YUV andluminanceconsidered harmful: A plea for precise terminology in video ".
  3. ^Jennings, Roger; Bertel Schmitt (1997)."DV vs. Betacam SP".DV Central.Archived fromthe originalon 2008-07-02.Retrieved2008-08-29.
  4. ^Wilt, Adam J. (2006)."DV, DVCAM & DVCPRO Formats".adamwilt.com.Retrieved2008-08-29.
  5. ^Clint DeBoer (2008-04-16)."HDMI Enhanced Black Levels, xvYCC and RGB".Audioholics.Retrieved2013-06-02.
  6. ^"Digital Color Coding"(PDF).Telairity. Archived fromthe original(PDF)on 2014-01-07.Retrieved2013-06-02.
  7. ^MSX Licensing Corporation (2022)."The YJK screen modes".MSX Assembly Page.
  8. ^Niemietz, Ricardo Cancho (2014).Issues on YJK colour model implemented in Yamaha V9958 VDP chip(PDF).
  9. ^"VCFe Vortrag vom 2016.04.30 – Homecomputer und Spielkonsolen – Videoarchitekturen als visuelles Medium".neil.franklin.ch.Retrieved2022-11-13.
  10. ^IC Master.United Technical Publications. 2001.
  11. ^Martín Sesma, Sergio (2016-10-03).Arqueología informática: los ordenadores MSX en los inicios de la microinformática doméstica(Proyecto/Trabajo fin de carrera/grado thesis). Universitat Politècnica de València.
  12. ^Redazione (2008-10-20)."MSX – Vari Costruttori- 1983".CyberLudus.com(in Italian).Retrieved2022-11-13.
  13. ^"V9958 MSX-VIDEO TECHNICAL DATA BOOK"(PDF).1988.
  14. ^Alex, Wulms (1995)."Schermen op MSX – De 2+ schermen"(PDF).MSX Computer & Club Magazine(72).
  15. ^Poynton, Charles (2008)."Chroma Subsampling Notation"(PDF).Poynton.com.Retrieved2008-10-01.
  16. ^abcdenum AvChromaLocation,ffmpeg 3.1.
  17. ^"y4minput.c - webm/libvpx - Git at Google".chromium.googlesource.com.420paldv chroma samples are sited like:
  18. ^Munsil, Don; Stacey Spears (2003)."DVD Player Benchmark – Chroma Upsampling Error".Secrets of Home Theater and High Fidelity.Archived fromthe originalon 2008-06-06.Retrieved2008-08-29.
  19. ^"Support decoding yuv442 and yuv421 jpeg images. · FFmpeg/FFmpeg@387d860".GitHub.
  20. ^abcChan, Glenn (May 2008)."Toward Better Chroma Subsampling: Recipient of the 2007 SMPTE Student Paper Award".SMPTE Motion Imaging Journal.117(4): 39–45.doi:10.5594/J15100.
  21. ^abLarbier, Pierre (October 2015). "High Dynamic Range: Compression Challenges".SMPTE 2015 Annual Technical Conference and Exhibition:1–15.doi:10.5594/M001639.ISBN978-1-61482-956-0.
  22. ^"Gamma-correct chroma subsampling · Issue #193 · mozilla/mozjpeg".GitHub.
  23. ^"WebP: sharpyuv/sharpyuv.h | Fossies".fossies.org.Assumes that the image will be upsampled using a bilinear filter. If nearest neighbor is used instead, the upsampled image might look worse than with standard downsampling.
  24. ^Norkin, Andrey (27 September 2016).HDR color conversion with varying distortion metrics(PDF).SPIE Optical Engineering + Applications, 2016. pp. 99710E.doi:10.1117/12.2237040.
  25. ^"BT.2020: Parameter values for ultra-high definition television systems for production and international programme exchange".International Telecommunication Union.2014-07-17.Retrieved2014-08-31.
  26. ^Chung, Kuo-Liang; Liang, Yan-Cheng; Wang, Ching-Sheng (March 2019). "Effective Content-Aware Chroma Reconstruction Method for Screen Content Images".IEEE Transactions on Image Processing.28(3): 1108–1117.Bibcode:2019ITIP...28.1108C.doi:10.1109/TIP.2018.2875340.PMID30307864.S2CID52964340.
  27. ^Annotated Glossary of Essential Terms for Electronic Production.doi:10.5594/SMPTE.EG28.1993.ISBN978-1-61482-022-2.Archived fromthe originalon November 30, 2017.luma: To avoid the interdisciplinary confusion resulting from the two distinct definitions of luminance, it has been proposed that the video documents use luma for luminance, television (i.e., the luminance signal), and chroma for chrominance television (i.e., the chrominance signal)
  28. ^"H.273: Coding-independent code points for video signal type identification".www.itu.int.2016.NOTE – The term chroma is used rather than the term chrominance in order to avoid the implication of the use of linear light transfer characteristics that is often associated with the term chrominance. [...] NOTE – The term luma is used rather than the term luminance in order to avoid the implication of the use of linear light transfer characteristics that is often associated with the term luminance. The symbol L is sometimes used instead of the symbol Y to avoid confusion with the symbol y as used for vertical location.
edit
Retrieved from "https://en.wikipedia.org/w/index.php?title=Chroma_subsampling&oldid=1247524143"