Separates vs. receiver confirmation (1 Viewer)

[Ted Lee]

man...has this thread taken a serious fart or what?
this is an argument that (literally) cannot ever be solved.
the reason is simple...
regardless of how scientific a test is, doesn't it all boil down to whether or not the listener can discern a difference. of course, the answer is yes. and therein lies the ultimate problem with this experiment:
sound and listening is subjective. no one call tell someone else what they are/are not hearing. period.
so, regardless of how detailed the test is, ultimately we'll have to rely on human perception.
the problem here is that someone is criticizing (sp?) someone else's perception and the quality of their gear. who wants to hear that their 5k amp sounds no better than a 700 dollar receiver?
emotions fly high in the face of battle!
umm...what was i talking about?

 

[AjayM]

The problem is that every single blind test I have ever seen/read/heard about is flawed from the start. Now of course some people will say I'm just making excuses, etc. But if you really look at it, none of the tests were done anywhere near the conditions you would use to conduct a medical blind test.

For example, there was one test in a big confrence hall like room, this was a speaker wire test using an ABX box. They hook up the ABX box to source using 16ga wire, then on one side they use some expensive brand and on the other side they use more 16ga wire. Now call me whacky, but if you want to test a wire you should use the same wire from source to component. This would be like taking a DBT medical drug test for headache medicine and the first thing you do is take 2 asprin then the 2 mystery pills. Next the problem was of the room, I don't know about you, but I don't listen in confrence hall sized room that can hold a couple of hundred people. Next the article made no mention of the people that were able to tell a difference between the wires during the test. This comes from an article on Stereophile's website, somewhere in the archives (don't have time to get the link at the moment).

The ABX website is a great one. They don't list results or any kind of test conditions, etc. Show me the results of the individual, again in a medical study (Since everybody keeps telling me how the DBT is #1 for the drug market) you would have individal results. (this can all be found at the ABX website).

You want to know why you see these poorly done tests? It's because doing a "proper" DBT is going to be almost impossible to do for audio equipment. The Double part of DBT means that the person swapping out the equipment/wires/whatever doesn't know what equipment he is working with. Which means you have to remove anything that will give him any indication of what that product is. How can you do that realistically? Are you going to open up an amp and add 20lbs of lead because one weighs less than another? Are you going to wrap a bag around it (which will be real good for ventilation) and then change all of the connections on the back of it? Do you see where this could be a little much for somebody to do in order to audition or test a piece of gear?

If you don't believe different equipment can and does sound different, go to your local audio dealer, put on a blind fold and have somebody walk you to each room he has setup. If you want to cheat by saying you heard a difference when there wasn't one, that's fine, or vice versa, that's fine to, but you are only cheating yourself.

the problem here is that someone is criticizing (sp?) someone else's perception and the quality of their gear. who wants to hear that their 5k amp sounds no better than a 700 dollar receiver?

Well, that can be a part of it. But not all of it. I don't have a $5k amp. I have a $700 2/ch amp. And when I auditioned it, I didn't know anything about the maker of the amp, it was just another one the dealer offered (and -gasp- it sounded better to my ears than the other amp I was listening to, which was a popular name brand that was about a $100 more money!).

Andrew

 

[EricHaas]

"regardless of how scientific a test is, doesn't it all boil down to whether or not the listener can discern a difference."

Uh yes, but if the test is BLIND, in theory there should be no bias in the listener's perceptions.

" of course, the answer is yes."

NO. The answer is "maybe." That is the point of the test.

"sound and listening is subjective."

Yes exactly. That is why you have a double blind test.

"no one call tell someone else what they are/are not hearing. period."

Yes and no. If you mean "hearing" in the subjective sense, no. If you mean hearing in the objective sense, then the answer is yes.

"so, regardless of how detailed the test is, ultimately we'll have to rely on human perception."

The test, properly done, is about weeding out the subjective component, which you correctly call "perception," from the actual objective component of the sound.

"the problem here is that someone is criticizing (sp?) someone else's perception and the quality of their gear. who wants to hear that their 5k amp sounds no better than a 700 dollar receiver?"

LOL. Yep, that could explain why everyone disagrees with Jaleel, but no one has any data to back up their position!

 

[EricHaas]

"You want to know why you see these poorly done tests? It's because doing a "proper" DBT is going to be almost impossible to do for audio equipment. The Double part of DBT means that the person swapping out the equipment/wires/whatever doesn't know what equipment he is working with. Which means you have to remove anything that will give him any indication of what that product is. How can you do that realistically? Are you going to open up an amp and add 20lbs of lead because one weighs less than another? Are you going to wrap a bag around it (which will be real good for ventilation) and then change all of the connections on the back of it? Do you see where this could be a little much for somebody to do in order to audition or test a piece of gear?

If you don't believe different equipment can and does sound different, go to your local audio dealer, put on a blind fold and have somebody walk you to each room he has setup."

Good point the second paragraph. Seems to answer your concerns in the first. Don't they indeed blindfold people for these tests, or at least put the components behind a barrier where they cannot be seen? Sorry, but I do not agree with your assertion that a reliable double blind test is not possible. As to your statement that the tests that have been done are *actually* unreliable, I cannot comment, because no one seems to know where these studies are. I will check into the Stereophile archives on that one you mentioned though.

 

[RicP]

Yep, that could explain why everyone disagrees with Jaleel, but no one has any data to back up their position

Eric, Jaleel has provided no data to back up his position either. He relies on hearsay and anecdotal evidence. He has admitted that he's never participated in any type of DBT, and in fact made his determination of the similarity of two amplifiers based on the sound of a WAV file played on his computer!

In addition he refuses -- for some unkown reason -- to disclose exactly what equipment is in his own system. What is the reason to do that unless it would expose you as a hypocrite?

 

[Ted Lee]

hmmm...

i think i get what you guys are saying, but maybe i'm just not stating my thoughts too clearly.

i still don't get how listening can be considered objective in any sense?

assume a "perfect" dbt environment here:

1. i listen to sample A

2. i listen to sample B

3. i decide sample A sounds "better"

isn't that a subjective statement? to me, i'm expressing my opinion, not something that can be quantified (is that the right term)?

hope i'm making some sense. i just can't see how auditioning can (even in a perfect environment) be considered objective.

 

[Larry B]

The physical problems associated with audio equipment (e.g., the fact that amps are heavy) need not counfound a DBT. The important point of the DTB is the the listener and the EVALUATOR of the results be blinded. Accordingly, an individual could perform the switches, but not be allowed to communicate in any way with the subjects; the subjects would be notified as to which piece of equipment ("A" or "B") they were listening to by, for example, an electric sign. In this scenario the switcher would have no contact whatsoever with the subjects. The subjects would write down their impressions, which would then be evaluated by a blinded individual. Only after the evaluations were completed would the code be broken.

On a separate note, there is reason to believe that DBTs may not be the best way to evaluate audio equipment. The problem is not with its being DB per se, but rather, with the A/B nature of the test. This was addressed a few months ago in a well-written article in The Absolute Sound. The essence of the article, with which I agree completely, was that many properties of audio equipment are not apparent during short-term sessions but instead, reveal themselves only after extened listening sessions. (An examle of this is overly extended highs; while these often are perceived initially as enhanced detail, they can (and often do) lead to listener fatigue after extended listening.)

Larry

 

[EricHaas]

"Eric, Jaleel has provided no data to back up his position either. He relies on hearsay and anecdotal evidence. He has admitted that he's never participated in any type of DBT, and in fact made his determination of the similarity of two amplifiers based on the sound of a WAV file played on his computer!"

If you read my first post in this thread, you'll see that this response is unecessary. No offense taken, but do read my first post.

"In addition he refuses -- for some unkown reason -- to disclose exactly what equipment is in his own system. What is the reason to do that unless it would expose you as a hypocrite?"

Yeah, that IS peculiar. No one else hear considers the identity of their gear a trade secret. What gives Jaleel?

 

[Larry B]

Do any of the participants in thnis thread entertain even the slightest glimmer of hope that they will influence Jaleel's opinions? If so, keep arguing with him. If not, turn off your computer and go enjoy some music.
Larry

 

[EricHaas]

"isn't that a subjective statement? to me, i'm expressing my opinion, not something that can be quantified (is that the right term)?"

Correct, BUT... The point of a double blind test is not necessarily to prove which people like better. In the first instance, it is simply to determine if they can tell two components apart at all. After 20 repetitions of a test, if someone can reliably say "that sound comes from amp A and that sound comes from amp B" this would lend credance to the position that the type of amp can create significant sonic differences. If not, it lends credance to the other view.

What people like and don't like is of course more difficult to quantify. However, even THAT can be measured, if somewhat imprecisely, by evaulating trends. For example, have 100 people listen to amp A and amp B blindly with all other variables controlled as equal. If 90% prefer amp A, it does not of course mean that amp a is the superior amp. But it might indicate to someone reading the study that there is a larger chance of that person preferring amp A than preferring amp B.

 

[EricHaas]

Simple, your sample consists of people who claim to be "audiophiles." In particular, it should consist of people who claim to be able to make minute distinctions between mid-fi, high-fi, and ultra-fi gear. If the people that are claiming to be able to make the distinctions can't, then the hyposthesis has been disproven for all practical purposes. If there are a few platinum eared people who are better then these audiophiles who CAN tell the difference, well then all that's been established is that there are subtle differences that only gifted people can detect. Best would be if everyone could detect the differences of course. That kind of finding would be meaningful to the most people.

 

[Larry B]

LarryB"

As an example of the latter, how many people have owned a system that you thought was very good, upgraded, and then couldn't stand to listen to your equipment
anymore? And over time grew dissatisfied with the upgrade and decided to upgrade again? Nobody here I'm sure.

I made virtually the identical point in a thread here about 6 months ago. Great minds really do think alike.
Larry with-a-space B

 

[AjayM]

On a separate note, there is reason to believe that DBTs may not be the best way to evaluate audio equipment. The problem is not with its being DB per se, but rather, with the A/B nature of the test. This was addressed a few months ago in a well-written article in The Absolute Sound. The essence of the article, with which I agree completely, was that many properties of audio equipment are not apparent during short-term sessions but instead, reveal themselves only after extened listening sessions. (An examle of this is overly extended highs; while these often are perceived initially as enhanced detail, they can (and often do) lead to listener fatigue after extended listening.)

I agree 100% with this statement, and I'll refer to this story;
I've come to the conclusion that short term AB tests are pretty much worthless. I have one anecdotal story that illustrates this particularly well. A friend of mine and I compared his $4000 Audio Research preamp with my $400 Audible Illusions preamp for an afternoon, swapping the two in and out of his system. We both agreed that the differences were subtle, perhaps too subtle to justify the staggering difference in price. One summer, he left for England and lent me his SP-10. I listened to it for a whole summer and really enjoyed my stereo. When he came back and I replaced the SP-10 with my Audible Illusions preamp, my system suddenly sounded lifeless and I didn't enjoy it as much. This puzzled me because on the AB test we both agreed the differences were subtle, yet the long term pleasure somehow wasn't even close.
I shortly thereafter went out and bought an SP-11 (the newest model at the time, listing for about $5000). When my friend sent his SP-10 back to the factory to be blessed or retubed or something, he borrowed my Audible Illusions, thinking that it would tide him over. I hadn't told him how disappointing I found it after the SP-10. He called me up a few weeks later to say that he was confused and that his system wasn't giving him any pleasure, that he was hardly listening to it. How could it sound so lifeless when an AB test proved nearly inconclusive?
An interesting question is why we ever thought an A/B test was so great to begin with. After all, would you evaluate two performances of Beethoven's Waldstein sonata in 1-minute snippets? If there were a magic machine that could zap you into an Acura NSX or a Chevy Caprice for two minute intervals, would you test the cars that way or would you live with each one for a week? And what about mates? Have you ever had a perfectly pleasant date with a person only to discover that marriage to the very same person wasn't such a great idea?
Again far from scientific and will get immediatly shot down as not being a "proper level matched DBT", but is interesting none the less.
Andrew

 

[Mark Austin]

Steve,
Sorry I didn't see your post about this.

Quote:

What would it tell us if they did distinguish between the two? Nothing. It tells us there was a difference. It doesn't give us any information beyond that.

 

[RicP]

An interesting question is why we ever thought an A/B test was so great to begin with. After all, would you evaluate two performances of Beethoven's Waldstein sonata in 1-minute snippets? If there were a magic machine that could zap you into an Acura NSX or a Chevy Caprice for two minute intervals, would you test the cars that way or would you live with each one for a week

This is exactly correct and exactly how I feel on the topic at hand.

Ajay,

I've had a very similar experience with not noticing much of a difference whether or not my power conditioner was affecting the sound. I decided to leave it in place anyway, figuring that it's surge protection was worth it even if there were no sonic improvements.

Well, 3 months later when I removed it to test a ground loop issue, I was absolutely floored to hear how much poorer my rig sounded wen I removed it, even though I couldn't hear a difference when I first installed it and did an A/B test.

 

[Larry B]

AjayM:

Of course this is a way to do it. But this is no longer a double blind test, it is now a single blind test. A single blind test will get shot down by more than a few people as being un-accurate.

I believe it IS double blind because neither the subjects, nor the evaluators of the data, know which piece of equipment is which. In the scenario I described, the person switching the gear is irrelevant since he/she has no contact with the subjects.

And thanks for sharing your story about the amps. I wish we could get more people to understand the significance of this phenomenon.

Larry

 

[AjayM]

I believe it IS double blind because neither the subjects, nor the evaluators of the data, know which piece of equipment is which. In the scenario I described, the person switching the gear is irrelevant since he/she has no contact with the subjects.

Here's a lose definition of the tests,

"single-blind" is one in which the subject does not know which unknown he is judging at any moment. "Double-blind" properly refers to tests in which neither the subject nor the operator know which is which. In a double-blind test, the experimenter (different from the operator) has made provision for identifying the unknowns, but the operator is not privy to these arrangements.

Although I think in your setup you could call it a sort of double blind/single blind test. Since the operator does know what is being tested, but with no access to the subjects it would be hard to cry foul in the test to say that the operator is giving the "wink and the gun" to the test subjects/results taker. Either way, it's still semantics, because who is going to go to that trouble to do a test?

Andrew

 

[Brandon_T]

Since none of you will be needing your amplifiers anymore, I will gladly accept your donations. Please contact me on where to send them to. I will cover shipping.!! Enjoy your bookshelf systems.
Brandon:b

 

[Steven Simon]

Man,

After reading this post, I'm thinking of trading in my Sub-Par gear, and going for that awesome Sony Home Theater In a Box !!!!

Screw Parasound Amps, Denon Recievers, Paradigm Reference Line Speakers, SVS Subs, and Progressive Sacn DVD Players!!!!!

I SAY WE ALL JUMP SHIP!!!!!

 

[Legairre]

Oh Yeah, you guys will be able to find all my gear on Ebay tomorrow with no reserve limit. Just take it. I don't even want good stuff anymore. I saw an Aiwa bookshelf system at KMart today for $269.99. It claimed to be a 5.1 system that decodes DD. After all the Aiwa amps and the Rotel amps all sound the same.