Robert Harris on The Bits - 8/3/04 column - OFFICIAL THREAD (1 Viewer)

[Michel_Hafner]

Some more remarks:

While EE is added after or during downconversion from HD to SD there is also already EE (high frequency boosting aperture correction) on many HD transfers themselves.

On full resolution displays/projectors you can see it. It's usually much less obvious than on DVD due to the higher resolution, but it is there. For example practically all Sony Columbia-Tristar HD versions I have seen have frequency boosting which is visible. It is common to sharpen HD or DI.



One of the main reasons we still see bad looking DVDs again and again, even from top studios and of top titles, is inadequate monitoring hardware. It makes quite a difference if you see the result on smallish studio monitors or projected wide to 2-3m. What often looks nice or at least acceptable on the studio monitor falls apart when magnified a lot more times. The lack of detail is suddenly very obvious, what looked nicely crisp around edges now looks butt ugly with large haloes. Ringing not visible before is visible. Compression noise, flickering of pixels, smearing is now in your face.

These things are visible on the monitor too, if you know what to look for and go close enough. But why should the compressionist care if he has to stick his nose to the glass? The consumers sit several screen heights away and won't see it, right? Wrong, if you have a large screen and/or use good quality projection (and more and more people do). It has not yet dawned on these post facilities, I guess, that the DI, HD and DVD should be quality controlled on the same high res projector or monitor to make sure they look the same as much as the formats allow. Except for loss of resolution, unavoidable compression issues and the different color spaces they should look the same. But they rarely do.

 

[TedD]

Apparently, another DVD needs to be added to "disaster" category: Paramount's "Star Trek: Generations".



Based on another thread here at HTF, it has a number of quality issues. It's looking like a recall candidate at this point, but nobody will say why it's being recalled. (There are also some issues with the content on the disc not matching the content described on the back cover (missing trailers)).



I don't have it, so I won't be posting any examples.





AMEN!!! Brother. Until this problem is addressed, we will continue to see many sub-standard DVD's.

Ted

 

[Eric Stewart]

Greetings, all.



RAH … let’s hope Ron Epstein doesn’t think this thread has — chortle — “elephantiasis.”



Ted … what exactly is a PNG file, anyway?



Ed St. Clair … what’s wrong with “made in Mexico” (as long as it’s not bootleg)? Or did I miss a joke somewhere?



Christopher … amen! The filtering which is known as MPEG-2 compression is not avoidable for the nonce … But stay tuned for MPEG-4 and other schemes used on the coming high-def DVDs (but let’s hope there’s not a format war) … And didn’t Mr. Maloney mention something called “wavelet compression”? … Still, you’re absolutely right: “The question which we must answer is how we can consistently make the most effective use of” MPEG-2.



Michel … very interesting. I presume you see HD transfers on Digital-VHS, no? Or how? … The aperture correction that EE’s such a transfer: it’s done because the scanning spot is not an ideal dimensionless point and therefore softens edges, right? … Yes, “inadequate monitoring hardware” is a factor that should get much, much more attention than it does. I guess one big reason they use small monitors in post facilities is the layout of the console or bay they are accustomed to sitting at. It seems to me I remember from the “Obsessed by Vertigo” featurette that Messrs. Harris and Katz had access to a console that was spread before a huge film projection screen in a screening room. This, I think, was for the sound matching. But the general idea would seem to be that post facilities should use a similar arrangement for their video projects. Of course, just the dedicated space itself would cost money, not to mention the equipment. … And here’s a thought: why shouldn’t they invite sophisticated “preview” audiences to check out new releases before their final bitstreams are locked in?



Cheers,

 

[Robert Harris]

Eric,



The board you refer to is in the Hitchcock Theatre at Universal. One of the premier sound facilities in the world.



And I have no idea what all those little buttons and dials do. There are professionals for that.



But your point is well taken. There is seemingly no reason why a transfer or compression facility could not work with a large screen as in DI post.



RAH

 

[TedD]

PNG = Portable Network Graphics file. It uses lossless compression (zip) rather than lossy compression (JPEG).



Ted

 

[Ed St. Clair]

quote:Ed St. Clair … what’s wrong with “made in Mexico” (as long as it’s not bootleg)? Or did I miss a joke somewhere?

No joke.

I was "scared", only because I'd yet too see "MiM".

Any input?

 

[Michel_Hafner]

#Michel � very interesting. I presume you see HD transfers on Digital-VHS, no? Or how? � The aperture correction that EE�s such a transfer: it�s done because the scanning spot is not an ideal dimensionless point and therefore softens edges, right?



Yes, I watch HD from D-VHS tapes. I have about 300 archived.

The aperture correction is done to compensate for HF loss

in telecine. It can be done well or badly (with haloes

like DVDs). With DIs via film scanners there should be no

need at all to artifically sharpen, certainly not for DVDs.

There is plenty of real detail in theses images. The issue

is how to best filter down while keeping detail and avoiding

aliasing and compression problems. EE should not be applied

on principal.

 

[george kaplan]

quote:Seen a few disc with "Made in Mexico", recently.

Well that's just production facilities, and I don't think it's a problem. It's when they start outsourcing the film restoration and telecine operators to sub-sub miniumum wage workers in Malaysia that we'll be in trouble.

 

[Eric Stewart]

All,



Another issue that's been only lightly touched on in this thread: How could respected journals like Widescreen Review give the Cold Mountain DVD a score of 4.5 out of 5, saying it has "pleasing images with satisfying sharpness and detail, though some scenes have a slightly blurry appearance. ... Pixelization is noticed at times, creating a “digital” appearance, but edge enhancement is limited to what may only distract pickier viewers"?



Granted that there are offsetting plus-factors in the CM rendering on DVD, characteristics like an effective color palette which the review duly notes, it seems to me odd that a publication dedicated to making home theater "the best that it can be" would let the minus-factors off so lightly as being distractions to "pickier viewers" only.



It's not as if those who are complaining are absolute perfectionists. It's just that we all want home theater to indeed be "the best that it can be," given the many very real constraints such as how much data a DVD can hold.

 

[Eric Stewart]

I don’t know if anyone in this thread is up for yet more beating a dead elephant — uh, horse – but it occurs to me that we ought to recognize this truth …



On DVD, or in any other form of digital video transmission or storage, there is actually no “image”!



There are only sets of numbers which represent various relationships pertaining to a fictional “image” that has been derived from a real image on film. The numbers are decoded by circuits and algorithms in our homes. The relationships that the numbers represent are reproduced, after further processing for purposes such as deinterlacing and scan-rate scaling, on our display screens. What is on the screen is transferred to the retinas of our eyes. The retinas transmit signals to the brain, which interprets the signals. Then and there, and only then and there, does the digital “image” exist.



It’s not like with film. With film, the image is actually there, frame by frame by frame, trapped on celluloid. You can hold it in your hand. You can look directly at it and confirm that it is real. You can project it on a standard screen using a standard projector with a standard lens and a standard lamp, and know exactly what it looks like.



The digital “image” turns the film frames into numeric specks called pixels. That in itself changes it some. Reversing the film-to-video transfer operation would not produce film frames just like the originals. For one thing, quantization errors would introduce “noise.”



Anyway, then the numbers get crunched such that, for example, “the same image” can be reconstituted from a smaller set of numbers. That “image” will look even less like the original, true image on film.



But the digital “image” was never real. It cannot actually be reconstituted, because it was never constituted in the first place. It began life as a set of numbers that ostensibly had some relationship to a real (reel?) image on film.



Digitization and number crunching denature the real image into a fictional “image.” It’s the same sort of thing as audio purists say about CDs vs. vinyl. They claim they can “hear” the difference, and vinyl “sounds” more natural.



At least one contributor to this thread has pointed out that maybe the complainers about Cold Mountain and other recent Miramax DVDs ought to look to their own equipment. I think he errs in the opposite direction, but he does have a point. We talk as if the “image” on DVD is real, it is what it is, and if it looks a certain way, that’s it. DVDs don’t lie.



Well, maybe they do.



Film doesn’t lie, but maybe digital “images” do.



Case in point: I’ve been beefing about faces on DVD that have “color haloes” along the edges that they make with objects of contrasting tonality and hue.



I went through a lengthy analysis earlier in this thread and pretty well established (at least to my own satisfaction) that the culprit is 4:2:2 subsampling of the chroma using digital filters that are asymmetrical along the horizontal axis.



Does that mean that the “image” on DVD is stuck with these haloes?



Not really! Working with Ted D., I was able to determine that shifting the “chroma plane” one pixel leftward in the decoder did wonders for the problem as it plagues certain scenes in North by Northwest on DVD.



But one of the signal paths in my home does even better. While Ted’s one-pixel-leftward shift overcompensated to an extent, the chroma renditions of this DVD that I get using a DVI interface from my DVD player to my display are yet more accurate than that. The assumption is that relevant numbers are being crunched in a more elaborate way than just subtracting 1.



So does the “image” on DVD have color haloes, or not?



It’s like asking if a quantum entity is a wave or a particle.



The answer is yes!

 

[Robert Harris]

Very good point.



Which is precisely why entire mechanical and digital/software decoding systems should be checked every so often against some "known," which you feel has reliability and can be duplicated.



And since Mr. Stewart keenly made his point in regard to equipment quality and setup, I'm now wondering how many viewers who did not see specific problems may have had their sharpness control down in the minus area.



This alone would hide a multitude of sins.



RAH

 

[Bryan Ri]

If I may ask you a quick question Mr. Harris:



I was recently watching the featurette on the Rear Window disc, and on the interview either you or your counterpart stated that one of the problems with film restoration is that there are no standards to the final product produced.



That disc was released, I think, in 2001. Is that issue still the same, or have techniques developed to the contrary? Would you consider Lowry apart of this, or would they be a different entity in the restoration process all together?



Sorry if I misquoted you, and I loved your work on Rear Window, among other films.



Thanks,



Bryan

 

[David Grove]

Begin Devil's Advocate





Maybe the analog image isn't real either.



After all, an analog image is only ( at the micrometer by micrometer scale), a varying "clumping" of some silver-based chemical compound. The intervals between the clumps could be said to correspond to the sampling frequency, while the density of these clumps corresponds to the "pixel value" in the digital image. Of course, this process is very complex, and both the intervals and the densities are stochastic processes, and vary according to certain physical principles.



Again, at the micrometer level, the clumps are, in fact, discrete-- just like digital encoding. In other words, film is also a discrete (sampled) medium. The difference between film and digital imagery is in the nature of the sampling (interval, quantization, mapping of intensities to values [gamma], and other properties.), not the existence of sampling.



In fact, I believe it is possible, in the laboratory, to sample an "optical signal" (from a lens system, as opposed to from film) at a resolution finer than chemical-based film can produce.



(Sidebar-- It is also possible to scan (sample) film itself at a finer resolution than the film recorded (stored) the data. Example: consider the difficulties experienced by professional digital photographers when they scan film data at too fine a resolution.)





NOTE THE BIG "IFS"...



IF bandlimited signals are sampled properly, and IF no lossy filters or compression are applied, and IF the reconstruction is performed properly, THEN the reconstructed signal is indistinguishable from the original by any physical measurement. This is true, regardless of how may "generations" are produced.





I am just suggesting that it is possible that it may not be so bright a line between analog and digital signal processing. Neither one is an actual eyeball observing an actual scene. They both represent an encoded means to convey a message.









Disclaimer: I have a bias toward measuring phenomena by means which are independent of an observer's opinions, and which are repeatable.



Further Disclaimer: I am not intending to troll, and do not wish to start down a digital vs. analog "scorched earth path".



Even Further Disclaimer: I fully and truly respect the whole film based artform, and expecially those professional practitioners whose mastery of that craft is far, far beyond my understanding (of either the analog, or digital technology). Please continue to perform your magic, whether you practice in analog, digital, or hybrid. The proof is in the pudding, and I enjoy eating it, irrespective of the recipe.

End of Devil's Advocate





DG

 

[Robert Harris]

To Bryan Ri:



There are still no standards in film restoration, although there is some agreement between those working within the field as to some of the aspects.



Even within AMIA, the group to which most archivists belong, there has never been an agreed upon definition of what "restoration" actually is.



In regard to LDI... while they've done some very nice work, I have not yet seen anything that, to me, would fall under the category of actual film restoration.



My ongoing problem with LDI, is that they report that they have performed restoration services on films which have not been restored -- adding to the confusion between actual restoration of film elements and a clean-up of video images.



Should they actually break down their work into titles cleaned up digitally for DVD release and call them what they are...



as opposed to what they consider to be film restoration, and take a serious look at what the majority of film archivists consider restoration, their comments might be taken with a bit more seriousness.



RAH

 

[Bryan Ri]

quote:My ongoing problem with LDI, is that they report that they have performed restoration services on films which have not been restored -- adding to the confusion between actual restoration of film elements and a clean-up of video images.

This was exactly what I was curious about; thank you for your response.







Bryan

 

[Eric Stewart]

David Grove,



You're absolutely right about the analog image (that is, the image on film) not being real either ...



... except ...



... it does give you a "hard copy," or more than one (the oneg, the interpositive, the dupe, the release print, etc.) that you can hold in your hand and inspect and and measure deterministically ...



... except ...



... as Mr. Harris has so eloquently pointed out, when time and use have not been kind to these "elements" and they need to be resurrected. Once fixed up and transferred to digital video, they might in concept be thought to last forever ...



... except ...



... archivists are now trying to figure out a storage medium for digital information that won't grow obsolete and unreadable. Meanwhile, I've heard it said that dye-transfer prints a la Technicolor are real, real stable and maybe ought to get a second chance.



It's a complex world!



Cheers,

 

[ChristopherDAC]

Though I was hoping to avoid this, it looks like I am going to have to introduce the generally accepted performance standard for optical systems, Modulation Transfer Function. Although not a comprehensive measurement, MTF is applicable to any system -- direct viewing, film-based, video, or what have you.

In layman's terms, the MTF of a system is the plot of contrast ratio obtained on details versus detail size, measured at a fixed wavelength of illumination.

Most lenses have a MTF curve which rolls off gently like a low-pass filter, so that as details grow smaller they get murkier until some arbitrary cut-off point is passed beyond which the engineer says "no useful information". Film behaves in a similar fashion, although its MTF often varies strongly with respect to wavelength. Vacuum-tube video cameras and CRTs have something of this characteristic owing to factors such as beam spot size [Velocity Scan Modulation is an attempt to correct this, but I think introduces more problems than it solves], although the photosensitive coating on a modern video camera tube is fine enough that the brick-wall filter on the video signal typically cuts off the detail at a frequency such that it has not yet rolled off much.

A CCD focal-plane array, the typical technology in today's semiconductor-based video cameras, or an LCD, DLP, or LCOS array, is rather different. Disregarding the complications which arise when a detail of the same order of size as a detector element overlaps two elements, or the like, these devices have basically the ideal video MTF curve: ruler-flat all the way out to the limiting resolution, and then a sharp cutoff.

This is the explanation why film prints look worse after each generation: each optical printing step is the equivalent of passing the image through one or more gentle low-pass filters, so contrast and detail are gradually removed. Again, high-definition video looks razor sharp in close shots, but seems to lack detail in long shots [compared to 70 or well-done 35mm film], because it has limited detail but the detail which is there, is all there [remember that it will have passed through typically only one or two lenses, and at worst two vacuum-tube steps; while a film may easily have passed through a dozen optical elements to get to the internegative stage, depending on effects shots &c.]. This is also the reason why film prints from a HD source look so bad after the 1st generation or so: the LPF operation takes away that away that sharpness, and so the limited detail stands out.

As I have described it, the MTF is an unbiassed benchmark which can be used to describe any image transmission chain, whether optical, electronic, analog, digital, real, virtual, or whatever. So, there really is a basis of comparison between formats and display devices, no matter how odd things may get.

This MTF roll-off, by the way, is also experienced in telecine systems [as well as the pick-ups of optical disc players] and is the reason for the real edge enhancement, also known as aperture correction. This is simply an active filter which is supposed to boost amplitude of the high frequencies of the video signal [equivalent to fine detail] by the same degree to which the imperfections of the telecine equipment have reduced them. It is a rather gentle correction, and I hate to see it blamed for the evils of some bizarre MPEG or other video-processing artefact.

 

[David Grove]

ChristopherDAC refers to "inverse filtering", which, is the signal processing operation that, to the extent that a degadation can be precisely characterized (and this is frequently very difficult to do in signal processing practice because of noise and dynamic range limitations), results in the prescise (almost surgical precision) "undoing" of the degradation.



"Signal processing" in the above paragraph can be very general, encompassing analog, digital, and optical (another branch of signal processing which has its own magic) signal processing.



It goes without saying that, ultimately, onverse filtering (the reversal of degradation to the original signal) is the whole goal of restoration, regardless of the tools or technology employed.



DG




Edit for clarity.

 

[Michel_Hafner]

Mr. Harris, could you please elaborate on restoration

versus cleaning up video? What minimal requirements are there for technical procedures to be called restoration?

It's obvious that cleaning up a video master in NTSC or PAL

resolution, no matter how well done, is not film restoration. One can not take that master and make 35mm quality prints for theatrical distribution. So what are the criteria?

- A new restored negative or positive from which 35mm quality material can be printed?

- Restoration versus preservation versus 'better' than the original?



Lowry work now also with 2K masters, 4K is in the pipeline if not done already. Is a 2K or 4K 'restored' negative clone or clone of the best survining elements an acceptable restored 'film' master? Or has it to be output on negative again? What about the future where digital projection will be the norm and film projection an anachronism from the past? Will the archival negative become an anachronism too? Or on the contrary even more important considering how volatile digital data can be long term?

 

[Robert Harris]

To Eric Stewart...



I don't believe that I've ever reported that motion picture elements, copied to a digital base, are then protected forever.



This is not true.



Motion pictures have survived for some 110 years for precisely the opposite reason... that they can be played back on simple equipment.



Unfortunately, the digital world works in completely the opposite direction.



RAH