Robert Harris on The Bits - 8/3/04 column - OFFICIAL THREAD (1 Viewer)

[Eric Stewart]

Michel,



Let me express my appreciation to you for contributing most valuably to this discussion. I have a question about your essay on digital noise reduction at http://www.cinedrome.ch/hometheater/dvd/dnr/text.html.



Whereas I fully understand and agree that the various kinds of ugly motion artifacts that you list and document can and do derive from faulty DNR, I'm not quite sure how one can distinguish them after the compression stage from similar artifacts that can be introduced during compression due to poor motion estimation etc.



In other words, is there any way to look at (say) the Cold Mountain DVD and tell what aspects of the generally crappy appearance Mr. Harris complained of came from DNR and what from compression? Is there any way, that is, short of directly comparing the HD master, compression master, and final DVD?



Thanks.

 

[Michel_Hafner]

It depends on the type of artifact you see. There can be

cases where it's not clear if MPEG or DNR are causing them.

But MPEG artifacts are not very similar to DNR artifacts

in most cases. MPEG can cause blocking, ringing, mosquito

noise, I frame pumping and general noise on top of the

picture. It does not cause flickering or DNR type smearing

unless the MPEG encoder has built in DNR. With low

bit rates floating similar to DNR is possible, I guess.

So some cases are ambiguous without further knowledge

about the encoders and processors used.

I can't comment on Cold Mountain in detail since I have

not looked at it in detail. I don't know if DNR was used

on that DVD.

 

[Eric Stewart]

Michel,



By "ringing," do you mean "a blurring, or out of focus effect around the edges of an object that is moving from frame to frame [and that] occurs more often when there is a large amount of motion between frames of the video" (see http://www.dvdtech.com.au/services/video.htm?

 

[Michel_Hafner]

With ringing I mean layers of ghost edges due to removal of

high frequency components (some poster called it Gibbs phenomenon). It does not look the same as layers of ghost

edges caused by motion smearing of DNR.

 

[Eric Stewart]

Ah, yes, that would be David Grove, who wrote in post # 57 of this thread:




quote:I wonder if the low-pass filter may have been implemented as a "brick wall" filter in the frequency domain (in other words, by just zeroing out all DCT coefficients above the cutoff). Implementing "perfect" filters in that manner does knock out the intended frequencies, but has the notorious side effect of introducing ringing artifacts (due to Gibbs phenomenon).



Bjoern Roy shows an example, the NTSC Region 1 version of Lord of the Rings/Fellowship of the Ring, at [url=http://www.videophile.info/Review/FOTR/FOTR_01.htm]http://www.videophile.info/Review/FOTR/FOTR_01.htm[/url]. It's about 1/3 of the way down the page, a detail of the edge of a roof and wall with faint ringing, where the PAL version was fine.



I think this effect looks just like edge enhancement. (Does anybody agree or disagree?) But it comes from the very same "brick-wall" filtering intended to reduce horizontal detail and image "entropy" so the MPEG compression will have fewer problems!

 

[Eric Stewart]

While we're on the subject of problems with DVD picture quality ...



Would anybody care to comment on a problem I call "color haloes at edges of faces" ... see the web page I created about it at:



http://home.comcast.net/~epstewart/Test_Web_Page_1.htm



I have a 61" DLP rear projector that has a big enough screen to show up problems like this. At first I thought the problem was with the TV or the DVD player hooked to it, but the same problem shows up in my computer's software DVD Player ...



... so I have to assume the DVDs are encoded this way. This is not a rare problem. It appears to be pervasive ... even ubiquitous. Any time there is an edge transition from something of a contrasting color to a face, there is apt to be a "color halo."



Can any of you experts explain it?



Thanks ... it's been driving me buggy.

 

[ChristopherDAC]

I think the "colour halo" phenomenon probably lies with the 4:2:0 chroma subsampling. Simply put, one chroma value determines the tint of four adjacent picture elements, while the luminance values change in every element. Thus there will inevitably be colour bleed or blending at the edges of contrasting-colour areas. Now, every television system allows something like this because the human eye is fairly insensitive to spatial variations of tint, but that fact is considerably more true for a gradual gradation than for a sharp edge. Oddly DVD has more horizontal colour resolution than the old LD and broadcast NTSC format, but less vertical.



In terms of edge enhancement/ringing, the MPEG compression system does not just remove high frequency components in the video: the heart and soul of the lossy DCT approach is that it removes the low amplitude components of whatever frequency. This causes false picture texture [moire pattern, Gibbs effect edge haloing] by design but normal scenes are considered busy enough that the effect is not noticable. Take a look at the ATSC website for details, it discusses MPEG-II very thoroughly. The point is that, part of the "look" of a film is determined by the distribution of spatial frequencies in the image [I will expand upon this highly involved topic only if someone wants]; while as the bitrate decreases, all images will exhibit more and more false picture texture, the spectral power distribution will determine which images exhibit more or less of it. In the limit, a regular pattern of black and white comprising only a single frequency component would transmit cleanly at a virtually zero bitrate, while "picture snow" [totally random noise] could scarcely be compressed at all without visible flaws -- visible only, however, to someone who knew what it was supposed to look like! All real scenes fall somewhere in between -- they are significantly nonrandom, allowing compression, but also allowing us to pick out the flaws.

 

[Eric Stewart]

Christopher,



Thanks for clearing up what causes “color haloes at edges of faces” on DVDs. As I understand 4:2:0 chroma subsampling, it halves color resolution in both the horizontal and the vertical direction, leading (as you say) to color bleeding/blending at sharp edge transitions in either direction (or both directions). So, because of 4:2:0 subsampling, MPEG-2-encoded DVDs have less vertical color resolution (though more horizontal color resolution) than laserdiscs and NTSC. Interesting.



At any rate, this sounds like a problem inherent to MPEG-2 which could not be avoided by more careful encoding/compression. That makes it unlike the problems with edge enhancement, reduced high-frequency detail, etc. that have mainly been discussed in this thread. Do you agree?



As for your other comments re: MPEG-2 …




quote: In terms of edge enhancement/ringing, the MPEG compression system does not just remove high frequency components in the video: the heart and soul of the lossy DCT approach is that it removes the low amplitude components of whatever frequency. This causes false picture texture [moiré pattern, Gibbs effect edge haloing] by design but normal scenes are considered busy enough that the effect is not noticeable.



… I for one would like to hear more. I could not locate the thorough discussion of MPEG-2 you referred to at the ATSC website, but I do have one in Charles Poynton’s book, Digital Video and HDTV: Algorithms and Interfaces. As I understand it:



(1) Each 8x8 block of luma (gamma-corrected luminance) samples is fed into a Discrete Cosine Transform (DCT), an equation which produces an 8x8 matrix of output values called “DCT coefficients.”



(2) Because most scenes have relatively little information at higher spatial frequencies — i.e., not all that much fine detail — the higher-amplitude output values tend to cluster toward the upper left corner of the output matrix, representing the lower spatial frequencies. The output values associated with higher spatial frequencies are located in the other quadrants of the output matrix, and they tend to be small negative or positive numbers.



(3) Because of this fact, it takes fewer bits to represent the output matrix than to represent the input block, and some compression has already been achieved. At this point, however, the compression is not lossy: reapplying the DCT to the output matrix will yield the original block of luma samples, intact.



(4) But now the DCT output matrix is quantized. An 8x8 “quantizer matrix” provides divisors for the corresponding values in the DCT output matrix. Each value is divided by its divisor, and the quotient is rounded to a whole number, thereby throwing away the fractional part of the quotient. This is the only step of the process that makes MPEG-2 compression lossy.



(5) The result of the quantization step is another 8x8 matrix of values, but now — because values in the quantizer matrix are biased to do so — most of the values outside the upper-left, low-spatial-frequency quadrant tend to be reduced to zero. A zigzag scan of the 8x8 matrix of quantized values produces a linear string of values, the great bulk of which consist of zeroes at the tail end. These redundant zeroes can be “collapsed” into a much briefer representation, a simple “end of block” code. Also, there are typically sequences of zeroes embedded inside the remaining string, and they too (using “Huffman coding”) can be collapsed into single-number indications of their “run lengths.” None of these aspects of MPEG-2 compression is lossy.



(6) The upper-left term of the quantized matrix represents a “DC term”: the spatial frequency of 0 Hz, representing the overall lightness/darkness of the scene. It is often relatively large, and it is treated specially. Each DC term but the first is coded as a difference in value from the first in the series. This aspect of MPEG-2 compression is not, however, lossy.



(7) The MPEG-2 decoder in the DVD player reverses the order of these steps. When it arrives at the inverse quantization step, it knows what quantizer matrix the encoder used, and it uses the same one. Now, each value in an 8x8 matrix is multiplied by its corresponding value in the quantizer matrix. But because the fractional parts of the quantization computations were thrown away by the encoder, the decoder’s output from the inverse quantization step doesn’t precisely match the encoder’s original matrix of DCT coefficients. This “reconstruction error” is where the “lossiness” of MPEG-2 compression actually enters the picture.



All this seems to boil down to this: none of the aspects of MPEG-2 compression is lossy except for the quantization step. How lossy this step is depends on the values in the quantizer matrix. Higher values produce larger remainders that get thrown away. The larger the discarded remainders, the greater the reconstruction error.



The “typical” quantizer matrix has relatively large values in its lower right quadrant corresponding with high horizontal and vertical spatial frequencies (fine detail). It has relatively small values in its upper left quadrant corresponding with low horizontal and vertical spatial frequencies (coarse detail). And it has medium values in its other two quadrants corresponding with low horizontal and high vertical spatial frequencies and with high horizontal and low vertical spatial frequencies.



This is an arrangement which biases the quantization toward discarding high-spatial-frequency detail. It can even discard such detail in its entirety if this detail is low enough in amplitude.



But, as you so crucially point out, quantization can also wholly remove low-amplitude components of other frequencies. It is less likely to do so that with higher frequencies, simply because the quantizer-matrix divisors for these frequencies are not as large. But it can do so. Furthermore, when it doesn’t remove them entirely, it can impose rather gross DCT reconstruction errors on them.



Accordingly, the divisors in the quantizer matrix are key. If I read Poynton correctly, compression ratio control is effected by altering the quantizer matrix: higher values for more compression, lower values for less compression. The greater the compression ratios, the more “false picture texture [moiré pattern, Gibbs effect edge haloing]” there is.

 

[Eric Stewart]

After posting the above, it occurred to me that the MPEG-2 DCT/quantization process that I described for 8x8 blocks of luma (Y’) samples is also done for blocks of chroma (Cb, Cr) samples.



(In Y’CbCr, Y’ is the gamma-corrected version of Y, a linear luminance component. Where Y is linear, its Y’ derivative conveys changes in the image’s lightness whose values have been made nonlinear (gamma-corrected) to code for perceptual uniformity. Cb and Cr are the “color difference” signals that modify monochrome luma so as to “color the image in.”)



The MPEG-2 quantizer matrix used for Cb and Cr is different than that used for Y’. Poynton, in Digital Video and HDTV, p. 453, says, “Typically there are two matrices, one for luma and one for color differences.” (That sentence applies specifically to JPEG and Motion-JPEG compression, but I believe it carries over to MPEG-2.) The fact that luma and chroma have different quantizer matrices may, however, not mean a whole lot.



This may not either: because of the 4:2:0 chroma subsampling of MPEG-2 compression as it applies to DVDs, each 8x8 block of samples for chroma components Cb and Cr corresponds not to an 8x8 luma block, but to the four 8x8 luma blocks that make up a 64x64 luma macroblock (see Poynton, p. 480). Each individual chroma block matches up with a luma macroblock. Automatically, the vertical and horizontal chroma resolution would each be half that of luma.



But here’s the thing that may mean a lot: we have assumed that luma quantization produces odd effects at sharp “lightness edge transitions. ” So why shouldn’t we assume that chroma quantization produces odd effects at sharp “chroma edge transitions,” too?



Due to 4:2:0 chroma subsampling, what qualifies as a sharp “chroma edge” transition would surely be less abrupt onscreen than a sharp “lightness edge” transition. So maybe any odd quantization-error effects such as the Gibbs phenomenon that pertain to the chroma components of the signal would be relatively spread out, spatially speaking – and therefore more obvious.



In other words, maybe the “color haloes at edges of faces” problem of which I complained in an earlier post is (at least in part) a “color version” of the ringing artifact, or Gibbs effect, which we have said besets over-compressed luma.



Hence, maybe the fact of 4:2:0 subsampling is part of the explanation, and maybe another part is the presence of “odd effects” at color edges due to over-aggressive quantization.



I used a screen shot from the “North by Northwest” DVD as an example of “color haloes at edges of faces” on my web page at:



[url=http://home.comcast.net/~epstewart/Test_Web_Page_1.htm]http://home.comcast.net/~epstewart/Test_Web_Page_1.htm[/url]



But these “color haloes” are not as evident, if at all, in earlier parts of the movie as in the scenes which begin as Cary Grant gets on the train and meets Eva Marie Saint. (I’ll have to check this again, but that is my recollection.) Maybe the MPEG-2 encoder used a different, less aggressive chroma quantizer matrix for the earlier scenes.



It’s food for thought, anyway …

 

[TedD]

Eric, can you supply the frame #'s or time stamps for your screen shots? I'll see what I can do about reproducing the captures on my system.



Ted

 

[Eric Stewart]

Ted,



Your input will be most appreciated. Here are the elapsed time stamps in question:



North by Northwest: Chapter 16, elapsed time approx. 45:06, Cary Grant tells Eva Marie Saint the police are after him because he has “Seven parking tickets.”



The Producers/Special Edition, side A, widescreen version: Chapter 15, elapsed time approx. 47:58, Roger De Bris (Christopher Hewitt) says to Gene Wilder, “Quel dommage.”



A Room with a View/Special Edition, disc 1: Chapter 12, elapsed time approx. 51:21, with a gathering in his mother’s drawing room, Daniel Day-Lewis is listening to Helena Bonham Carter play the piano.



Note: all of my screen captures represent portions of the original image. They are not the entire image.



My orignal web page has been updated to reflect this information. You can access it at:



http://home.comcast.net/~epstewart/Test_Web_Page_1.htm



Thanks again. I'll be eager to hear what you determine.

 

[TedD]

Eric:

OK, what we have here is a misalignment of the chroma plane with the luma plane, also known as "Chroma Delay".



This problem is specific to certain titles, and is not an inherent problem with all DVD's. The fact that the chroma has less resolution than luma tends to mask this problem, unless it exceeds roughly .5 pixel width. In addition, the chroma can be filtered separately from the luma, and it appears that the chroma on NxNW is considerably softer than the chroma on TP.



Chroma shift can occur in both the vertical and horizontal dimensions which explains your third example.



NxNW is a real problem, because the chroma plane seems to float around a little bit. Agreeing with your asessment, I saw very little to no chroma delay in the early parts of the film.



The scene you used as an example clearly had the chroma shifted right by approimately 1 pixel:



[url=http://webpages.charter.net/tvdias/NNW_Chroma.png]http://webpages.charter.net/tvdias/NNW_Chroma.png[/url]



Correcting this with FFDShow chroma offset of X = -1 results in:



[url=http://webpages.charter.net/tvdias/NNW_ChromaX-1.png]http://webpages.charter.net/tvdias/NNW_ChromaX-1.png[/url]



Which works for this section of the film, but causes issues in the early parts of the film by overcorrecting.



The Producers DVD is both better and worse. Worse because it is a 2 pixel shift, but better because it appears to be consistent throught the film.



Correcting this with FFDShow chroma offset of X = -2 results in:



[url=http://webpages.charter.net/tvdias/Prod_X-2.png]http://webpages.charter.net/tvdias/Prod_X-2.png[/url]



Unfortunately, since you are using a MAC, these tools are not available to help you with this issue.



So, in summary, what we have here is yet another correctable defect that is probably masked by the lack of high resolution non-interlaced display devices scattered throughout the transfer and mastering facilities used to produce these titles.



To be absolutely fair, however, many devices in the DVD playback chain can also contribute to the issues of chroma delay



I hope this helps.



Ted

 

[Robert Harris]

I continue to check this thread, which not only becomes increasingly more interesting, but speaks volumes about the "typical" HTF subscriber and contributor.



I have placed two calls to Mr. Maloney, but have not heard back from him. I'm hopeful that he is experimenting and attempting to better the mechanism by which DVDs are encoded and compressed and has not been pulled off the scent by a management fearful of publicity, which in this case can be only positive.



RAH

 

[Ed St. Clair]

quote:speaks volumes about the "typical" HTF subscriber and contributor.

These are NOT "typical" S&C's, RAH!

Check the Showgirls SE thread!

Just joking, guys!

Yeah, this is why HTF is The Best!

 

[TedD]

RAH, You have a PM.



Ted

 

[Michel_Hafner]

Ted,

How do you think chroma delay gets on a DVD like NBNW

which was digitally restored by Lowry Digital? Assuming

they added no delay (reasonable assumption) and the digital files were used for the DVD (also reasonable), where did

that chroma delay happen?

 

[TedD]

quote:How do you think chroma delay gets on a DVD like NBNW



No idea, when it comes to commercial DVD's, I'm just a consumer, not a creator. However, I'm pretty sure it wasn't done on purpose.



Only Lowrey would be equipped to answer that question.



Ted

 

[Eric Stewart]

Ted,



Thanks a million for your analysis of "chroma delay" on the DVDs I cited at:



[url=http://home.comcast.net/~epstewart/Test_Web_Page_1.htm]http://home.comcast.net/~epstewart/Test_Web_Page_1.htm[/url]



I want to digest what you said before responding in detail. But a question:



The "chroma delay" of which you speak is a sort of "supercategory" of "chroma delay" or "Y/C delay" as often referred to in discussions of analog NTSC video, no?



In the latter, the assumption is that the delay is actually a discrepancy in timing the luma and chroma components' separate processing paths. I.e., the electronics in a TV which process chroma may be slightly out of step with those that process luma.



But "chroma delay" or "chroma shift" as you speak of it can come from a variety of other sources, such as improper registration of the luma and chroma components being processed in the digital domain -- correct?



Again, Ted, thanks for your valuable inputs to this most interesting discussion. I agree with Mr. Harris that this kind of discussion is the real deal -- and offer my thanks to him for provoking it at the outset.



BTW, I'd like to ask Mr. H. if he is located near Mr. Maloney, who is apparently in Hollywood. If so, maybe Mr. H. could get together with Mr. M. at his post facility and the two of them together could settle a lot of the issues with which much of this discussion began.

 

[TedD]

quote:But "chroma delay" or "chroma shift" as you speak of it can come from a variety of other sources, such as improper registration of the luma and chroma components being processed in the digital domain -- correct?



Well, since FFDShow is capable of correcting it in the digital domain, I would have to say it can also be screwed up in the digital domain.

I have to apologize for not being a digital video theory uber geek like Eric or ChristopherDAC, but math has always been one of my poorer subjects.

On the other hand, 40+ years of working on computers and networking certainly has honed my problem determination and logical deductive reasoning skills.

Ted

 

[Eric Stewart]

Ted,



First, thanks again for your input re: “color haloes on DVDs” (see post #86 in this thread).



Alas, I’m not really much of a “digital video theory uber geek” yet. Actually, I think of you, ChristopherDAC, and several others in this thread as much farther along than I am. All I can really say for myself is that I’m trying to learn more than I presently know! Please consider this whole belabored discussion my own way of “honing my problem determination and logical deductive reasoning skills.” Like you, I find doing so pays handsome dividends.



BTW, I am also a (now retired) computer professional. My specialty was supporting mainframe operating systems, but I rubbed shoulders with plenty of networking gurus.



I am now in full sync with you: chroma delay, shift, or misalignment can occur in the digital domain, not just in the analog domain. Actually, I consider this a very insightful point that you make … because I have yet to find any mention or admission of this fact on the part of the real, professional digital video theory uber geeks!



Poynton, Digital Video and HDTV, p. 514, does say, “The Y’ [luma] and C [chroma] components should be time-coincident within ± 25 ns. Error in chroma timing is known as chroma-luma delay.” But that’s with reference to analog 480i NTSC composite video.



Now, moving on, here is a fuller response to your post #92 in this thread concerning chroma delay or chroma shift – which you call “a misalignment of the chroma plane with the luma plane” in the rendering of a film or other program on DVD. I agree with you that that is an excellent description of the problem.



Still, I have to think there is more to understanding this phenomenon than we have yet considered.



Before getting into that, a very minor point: did you notice that your “X – 1 pixel” FFDShow chroma offset for North by Northwest at …



[url=http://webpages.charter.net/tvdias/NNW_ChromaX-1.png]http://webpages.charter.net/tvdias/NNW_ChromaX-1.png[/url]



… fixed Eva Marie Saint up just fine, but now there is a red halo to the left of Cary Grant’s face?



OK, now, on to the main topic.



For ease of reference, here is my original web page documenting the “color haloes” problem:



[url=http://home.comcast.net/~epstewart/Test_Web_Page_1.htm]http://home.comcast.net/~epstewart/Test_Web_Page_1.htm[/url]



I have yet to encounter an actual DVD in which the objectionable color halo is to the left of and/or above the location where the chroma plane “should be.” The chroma shift is typically rightward and sometimes (or always?) downward.



So whatever produces the chroma shift is biased to do so in a rightward direction, and sometimes (or always?) also in a downward direction. The direction of the shift is apparently not random.



The first thing that accordingly comes to mind, as ChristopherDAC said in post #87, is 4:2:0 chroma subsampling. Interestingly, though, each MPEG-2 chroma subsample is nominally sited with a slight, half-pixel leftward offset, in comparison with the 2x2 array of luma samples it goes with. (I get this from Poynton, p. 90.) We’re trying to explain a rightward 1- or 2-pixel chroma offset …



… and also frequently a downward one of approximately the same magnitude. Nominally, MPEG-2 chroma subsampling does not involve any vertical shift whatever.



Conclusion: 4:2:0 subsampling is probably not, all by itself, the culprit here.



Obviously, something else must occasionally happen during the processing of the video signal to alter the direction and magnitude of the chroma offset or shift that MPEG-2-style 4:2:0 chroma subsampling nominally imposes. We don’t yet know what that something is.



That’s about as far as I can take it, at this point. In closing, let me echo what you said about :



• The chroma shift in NxNW does seem to wander around from section to section of the movie, while in The Producers it is constant. Interesting.



• The chroma in the former is “softer” than in the latter. I believe the former was “digitally restored.” Maybe that explains the difference.



• The “color haloes” defect is “yet another correctable defect that is probably masked by the lack of high resolution non-interlaced display devices scattered throughout the transfer and mastering facilities used to produce these titles.” Emphasis on “correctable.” (It would probably not be correctable if it were due solely to 4:2:0 chroma subsampling.)



Thanks again, Ted.