Digital in the signal path is bad? Really? A look at something that happened.

Chu Gai · 07-08-2003, 12:33 AM

A lot of people don’t like digital and prefer to run straight analog even going so far as to use vinyl. That is, of course, their prerogative to make that choice. Others are quite certain that any sort of AD-DA conversion is to be avoided because it leads to audible degradation.

But how much merit is there to this? Many of us have performed our own tests on equipment of our choosing and haven’t been particularly careful with regards to either level matching or paying attention to the elimination of biases. As a result, our findings can be skewed one way or the other, perhaps affected by our own predisposition to one philosophy or another.

The article below, which occurred about 20 years ago recounts some careful work that was designed to test this. The test was conducted in what would have to be considered most favorable for anyone who wants the ‘skeptic’ to be made a fool of or proven wrong. The person who set out to prove the inadequacy of digital was Ivor Tiefenbaum, who is presently (?) the managing director at Linn. He is, as is often said, a respected man and designer. The test was conducted at a place of their choosing using their equipment.

What’s of particular interest to me are the findings where it strongly suggests (I won’t say proves) that this test had a bit of cheating in it. Read it and you’ll see what I’m talking about.

Perhaps this will make some rethink their positions or rethink the pains that are taken in order to arrive at meaningful results. I certainly hope it’ll make you think that often times, the results that you see or hear when going to demonstrations are not necessarily because a product is better but because it’s been arranged that the chips fall a particular way. Sometimes you know, the Pepsi Challenge, Monster demonstrations, audible cable differences, happen in a particular way because of something pulled out of Houdini’s book.

Things are seldom what they seem, skim milk masquerades as cream…and all that.

Quote:

Digital Challenge: A Report

by Stanley P. Lipshitz (University of Waterloo, Waterloo, Ontario)

(From The BAS Speaker, Vol. 12 No. 11/12. Subscription to the magazine is the same as membership in the society, and can be had by sending $25 for one year ($35 outside North America) to The Boston Audio Society, Department C, Box 211, Boston MA 02126-0002.)

Some readers may be unaware of the background to the "digital test" in which Ivor Tiefenbrun participated on February 23, 1984 [1-4]. To summarize briefly: Tiefenbrun has been quite outspoken about the inadequacies of digital audio recordings and the systems on which they are made, and his organization, Linn Products Ltd., was instrumental in publishing an analysis of the Sony consumer PCM-F1 digital audio adaptor [5] outlining their objections. I, on the other hand, have been using this very system for a number of years now and have made close to one hundred recordings with it with superb results and not the "execrable results" reported by Tiefenbrun. I and my colleague John Vanderkooy have moreover conducted blind listening comparisons between the PCM-F1's input and its reconstructed output signals, and had yet to find anyone who can reliably distinguish between them on musical program material.

I therefore challenged the "anti-digital" community in general, and Tiefenbrun in particular, to participate in a blind listening test of the PC M-F1 to give them the opportunity to substantiate their claims of poor sound. The challenge details were spelled out in [2]. When I learned in February 1984 of Tiefenbrun's impending visit to Toronto, I reissued this challenge and was pleased to have him accept.

The test took place in the home of the local Linn distributor, Mr. Michael Remington, using his all-Linn/Naim system (Linn LP-12 turntable, arm and cartridge, Naim NAC 32 pre- and NAP 250 power amplifiers and Linn Isobarik loudspeakers) and his choice of program material (all LP records). Vanderkooy and Alan Lofft, editor of "Sound Canada" magazine, were also present. The atmosphere throughout was cordial and more relaxed than I expected.

The day began with two brief tests of the Tiefenbrun claim that undriven transducers (digital alarm watches, telephones, headphones, or other loudspeakers) in the same room audibly degrade the sound quality -- a claim which forms the rationale behind their "single speaker" demonstration demand. Firstly, a digital alarm watch with piezoelectric "beeper" was held about 500 mm behind Tiefenbrun's head while he listened to the loudspeaker reproduction from his stereo seat on the couch, with the watch either fully exposed or clasped firmly between the palms of my hands. We were assured that the latter artifice would muffle any deleterious effects. This was thus a single-blind test: The testee did not know the covered/uncovered status of the watch at each trial, but the tester did know. A random series of 20 trials was conducted while Remington cued up the turntable (playing a female vocalist) on each occasion, as he did throughout the day.

Tiefenbrun's result: 10 correct responses in 20 trials, an outcome which shows no ability to discriminate between the two situations.

The second test, also single-blind, used a Linn "Kan" loudspeaker as the undriven transducer. Again the female vocalist was used as source material. The loudspeaker lay on the thickly-carpeted floor behind the listening couch. It was placed either on its side (the "uncovered" condition) or on its face (the "covered" condition) according to a random series of choices. Ten trials were conducted during which Tiefenbrun achieved a score of 5 correct out of 10. Again, this demonstrates no discrimination ability beyond what one would expect purely on the basis of chance. With these preliminaries out of the way (but just for safety all watches and headphones were left in another room throughout the entire day's proceedings), we proceeded to the main test of the day -- that of the audibility of the Sony PCM-F1 digital audio processor in 16-bit mode when inserted into the audio chain, as proposed in my original challenge.

The PCM-F1's analog-to-digital (D/A) converter output was looped straight through to the digital-to-analog (D/A) converter input by connecting "video out" to "video in". The complete encode-decode chain including the two low-pass anti-aliasing filters as well as the sample-and-hold circuits and the A/D and D/A converters was thus subject to the test.

In order to remove the F1's polarity reversal it had been internally fitted with a digital inverter chip (as used in the later non-inverting Sony PCM-701 model) to interchange the ones and zeroes and hence perform a polarity correction.

The PCM-F1 was inserted into the signal path between the preamplifier and the power amplifier by means of an A/B/X switchbox of our design (see [6,7]). The setup is shown in Fig. 1. When the switchbox was in the "A" position the PCM-F1 was inserted into the signal path, while the "B" position represented the "straight-through" configuration. The "X" position was a randomly selected choice made by the switchbox. For each trial "X" was chosen by the box to be either "A" or "B", but its identity was unknown to any of the participants, thus making the experiment double blind. The subject could at his leisure compare any of all of "A", "B" and "X" with each other. What he had to do was identify whether "X" was "A" or "B". Once his decision was made, the switchbox was interrogated to discover the true identity of "X", and then "X" was re-randomized in preparation for the next trial.

The gains of the "A" and "B" paths were matched in both left and right channels to within 0.05 dB at 1 kHz using the PCM-F1's gain controls. This was done by measuring across the amplifier output terminals. The match was then confirmed to be within +/- 0.25 dB across the whole audio band. The PCM-F1's "peak hold" feature was used to keep a record of the peak signal levels passing through it during the test, especially in view of the relatively high sensitivity of the Naim power amplifier (<1 Vrms at clipping) and the relatively low listening levels chosen by the participants. More about this shortly.

After an acclimatization period, a set of 10 trials was conducted in an unhurried fashion before breaking for lunch, after which a further set of 10 trials was conducted. Tiefenbrun's score for the series was 11 correct decisions out of 20, a result which shows no statistically significant ability to discriminate between "A" and "B" any more accurately than would be expected on the basis of random guessing.

At this point I thought that I could reliably distinguish between the "A" and "B" paths on the basis of the *slight* noise level increase which occurred when the PCM-F1 was inserted into the chain, and which was marginally audible due to the high gain of the Naim MAP 250 power amplifier combined with the low peak signal levels through the F1, which the peak-hold meters showed to have risen no higher than -20 dB. (0 dB is the digital clip point, and these peak levels were somewhat unfair to the digital processor since 20 dB of its signal-to-noise ratio was being thrown away.) [In other words, for this segment of the test the F1 was in effect a 13-bit processor. -- Ed.]

I expressed my desire to try the test, and Remington went to cue up the record again, but I requested to be allowed to undertake the test with no signal passing though the system. Before realizing the import of what he was saying, Vanderkooy interjected: "Ah! You're going to listen to the sound of the relays." Yes, there is indeed a slight audible difference between the acoustic "click" made when the "A" and "B" relays pull in. This is due to the unavoidable differences in the mounting positions of the relay on the A/B/X box chassis and, although slight, it can be heard if one listens for it. I replied that I was going to listen to the difference in background hiss, and the subsequent series of blind trials showed conclusively that the two signal paths could be reliably distinguished on this basis alone.

What conclusion can we draw from this? Tiefenbrun's random results show that he had not been aware of either the background noise difference -- masked as it most likely was during his trials by the record surface noise -- or the relay click difference. The null result of this first PCM-F1 listening test is thus valid in spite of the potential for non-blindness due to the subliminal noise differences. I had been aware of the hiss difference from the outset from listening while the stylus was off the record, but had decided to proceed with the test in the knowledge that if the test produced a null result, the hiss difference could not have affected the outcome -- a decision vindicated by the actual data.

The "secret" about the slight relay noise difference was, however, now out in the open, and if listened for could from then on influence (albeit subconsciously) the blindness of any succeeding tests made using the switchbox. Nevertheless we decided to proceed with a second series of PCM-F1 listening tests, this time with the digital system inserted into the preamplifier's tape monitor loop. Once again levels were carefully equalized and frequency responses checked to be very closely matched between "A" and "B". This insertion point precedes the preamplifier's volume control, where the signal levels are significantly higher, thus more fully exercising the F1's full dynamic range while at the same time rendering its output noise level inaudible at the replay volume setting then being used. This configuration removed the hiss difference as a potential factor.

A set of 23 double-blind trials was conducted listening to music through the system, with Tiefenbrun voting for the identity of "X" at each trial. It was then that Vanderkooy pointed out that, although the relays in the A/B/X box were switching normally during this series, no tiny electrical clicks were audible from the loudspeakers during their operation, as normally would be the case. Investigation revealed that the preamplifier's "source/tape" monitor switch had inadvertently been left in the "source" position, and as Fig. 2 reveals, the A/B/X box and digital system were thus not being inserted into the chain at all during this run!

Because the "A" and "B" positions were completely identical due to this error, an unbiased decision for "X" should have produced a 50:50 split of "A" and "B" votes. Interestingly, it turned out that this was not the case, Tiefenbrun's votes being 14 for "A" and 9 for "B". Could something have been influencing his voting, or did he just prefer the letter "A"? A comparison of the actual "X" choices showed that during this series Tiefenbrun voted incorrectly in 16 out of 23 trials, although the sound from the loudspeakers never changed. [This proportion of incorrect guesses would occur in a truly random situation less than 5% of the time. -- Ed.] What does this mean? We cannot say for certain, but the most logical explanation is that he was (perhaps subconsciously) voting on the basis of the relays' acoustic click difference which we had now revealed. If so, his remembrance of the sound of the A and B clicks was inverted. Be this as it may, the error we had made by leaving the tape monitor switch in the "source' position turned out to provide an interesting sidelight on the question of personal bias. It also illustrates one of the potential pitfalls of high-resolution blind testing, and the danger of jumping to conclusions before very carefully checking the test setup.

With the tape monitor switch now correctly set, a rather rapid series of 37 trials took place with Tiefenbrun voting. The results: 10 correct decisions out of 37, a result far worse than would have been expected by chance alone. What was influencing the voting? Was it the relay sound? We do not know, but this seems likely. It should be remembered that using the A/B/X box, a direct comparison between "A" or "B" or "X" is always available, so a series of trials in which there is a tendency as above to produce a consistent reverse identification seems to implicate some extraneous factor unrelated to, but correlated with, the loudspeaker sound. Tiefenbrun did very little switching during this series, leaving the A/B/X box set to the "X" position most of the time. It does indeed seem possible that the identifications were being made on the basis of the relay sound incorrectly remembered.

To wrap up the day's work, we conducted a final series of tests to assess whether the relay contacts in the A/B/X box could have been electrically affecting the sound in any audible way. For this single-blind test the A/B/X box was powered up and left permanently in one position, and the relay contacts inserted into the signal path by switching the preamplifier's monitor switch to the "tape" position. The switching sequence (relay contacts "in" or "out") was determined by a random number sequence. Thirty trials were conducted, interrupted three times to provide Tiefenbrun with a reference comparison of the "direct" sound versus the relay contacts. His score in identifying the presence of the relay contacts was 12 correct out of 30 trials. This of course also shows no statistically significant ability to identify correctly when the relay contacts were in circuit.

In summary, then, no evidence was provided by Tiefenbrun during this series of tests that indicates ability to identify reliably:

(a) the presence of an undriven transducer in the room,

(b) the presence of the Sony PCM-F1 digital processor in the audio chain, or

(c) the presence of the relay contacts of the A/B/X switchbox in the circuit.

The tests were conducted in an amicable rather than confrontational atmosphere, and the parties departed feeling that the day's work had been worthwhile. Further carefully-conducted blind tests will be necessary if these conclusions are felt to be in error.

I would like to acknowledge the friendly cooperation and assistance of both Michael Remington and John Vanderkooy in carrying out these experiments, and would like to take this opportunity to express to Ivor Tiefenbrun, the guinea pig of the day's experiments, our genuine admiration for having the sincerity and guts to put his professed beliefs on the line.

References

[1] S.P. Lipshitz, "Views", "HFN/RR", Aug. 1984, p.15
[2] S.P. Lipshitz, "Views", "HFN/RR", Sept. 1983, p.19
[3] I.S. Tiefenbrun, "Views", "HFN/RR", Jan. 1984, pp.19,21.
[4] I.S. Tiefenbrun, "Views", "HFN/RR", June 1984, pp.13,15.
[5] A. Orlowski, "Digital Sound: The View from Scotland", "HFN/RR", June 1983, pp.34-37.
[6] S.P. Lipshitz and J. Vanderkooy, "The Great Debate: Subjective Evaluation", "J. Audio Eng. Soc.", Vol 29, pp.482-491 (1981 July/Aug.).
[7] D. Clark, "High-Resolution Subjective Testing Using a Double-Blind Comparator", "J. Audio Eng. Soc.", Vol 30, pp.330-338 (1982 May).

Brett DiMichele · 07-08-2003, 01:00 AM

You expect me to read that? No way.....

I will just
belive what everyone tells me to! Much easier!

John Garcia · 07-08-2003, 01:32 AM

I read the whole thing. Very interesting, to say the least.

Michael R Price · 07-08-2003, 06:53 AM

That's a great article, although it won't stop me from improving my CD player (perhaps not audibly, though).

Has anyone blind tested the audibility of changing the passive components in speaker crossovers? I hear a lot of people claiming that putting in good components (costing up to $50 for a single capacitor, for example) made the sound a lot smoother, clearer, etc. Chu, what do you think of this one?

Chu Gai · 07-08-2003, 08:48 AM

I'll confine my reply to things like resistors, capacitors, inductors.

I think it's entirely possible to replace one component with another and change the nature of the sound. Whether this is good or bad is difficult to answer but one must factor in something called 'builder's ear' which is the tendency for people to be enormously satisfied with something they've created. If you recall the wonderful movie, Unforgiven you'll see an example of that in the house that Gene Hackman built.

When people replace things there are a couple of things that often aren't done.
1) They don't measure the component that's being replaced to determine what it's value was. Nor do they measure the value of the component being put in its place. Not all components are stable for an infinite amount of time. Sometimes their values change in but a few years and being that human auditory memory is not a rock, we don't even notice it happening until one day something just isn't right. Depending upon the environment the component is in such as the temperature and even vibration (think about the insides of a speaker cabinet) this deterioration can be accelerated. Also a particular manufacturer, say Motorola for instance, occasionally comes out with a batch of whatever that doesn't age gracefully. As Elvis Costello said, Accidents will happen, it's only hit and run.
Electrolytic capacitors are a known culprit here because they can dry out and it's entirely possible to find that the capacitance has dropped by a factor of 10 or 100. Plus or minus 5% is one thing. Minus 100% or 1000% and you can be pretty sure there's a profound difference in what you bought and what you now have. You just might've lost the top end of your speaker response or lost the low end on your CD player. Of course, in the latter case, you could always fall back to the digital.
So if one knows what type of components are likely to age unfavorably, where they're located, and has some skill and chutzpah to delve inside with snips and a soldering iron, they can often restore a component like a CD player to meet it's original specs. Confirmation that there may be a problem can always be done by burning test tones on a CDR and playing it while measureing the output voltage. Of course this takes the fun out of upgrading and putting a new look on our equipment that now looks a bit dated.

2) When one looks in an electronic parts catalog like Mouser, one finds a ton of different flavors of a particular component. Resistors for example differ in things like rated power, maximum working voltage, operating temperature, etc. They also can have other electrical properties that cause them to differ. Inductance for example. A wire wound resistor has a non-significant amount of that and were you to replace a resistor with neglible inductance with a wire-wound one, you would be changing what that circuit was designed to do.

It is a strange thing that compels people to replace good components with other good and even more expensive components. Most though would not tolerate this sort of behavior from their auto mechanic. Golly gee Mr. Good Wrench, thanks for taking my brake pads out that had 30,000 miles left on them and putting in these Raybestos ones. Yeah, that chick that advertises them sure looks hot on TV. $450 for the job...thanks!!! Can I have a calendar too?

**HTF Ads**

Michael R Price · 07-08-2003, 12:11 PM

"It is a strange thing that compels people to replace good components with other good and even more expensive components. "

This is exactly the thing I was talking about... for example, upgrading the crossover in a nearly new GR AV-1, JBL S38 or Klipsch RF-7... that would mean putting in inductors with fancy wire, polypropylene instead of mylar capacitors, etc. See this thread: http://www.hometheaterforum.com/htfo...hreadid=140533

The point I'm driving at is that any distortion caused by passive components such as those in speakers is known to be less than that of any active component, like a transistor... and then there are also dozens of capacitors and such in all our equipment. So if there really are differences in crossover components, there sure as heck are differences to be found in digital equipment.

RobertR · 07-08-2003, 12:21 PM

Good article, Chu.

I've read of a similar experiment done by the late Gabe Weiner (recording engineer), who had people listen to a direct mike feed of music being performed vs listening to it through a real time A/D D/A chain.

They couldn't tell which was which.