What's new

$10k golden ears AMP challenge! (1 Viewer)

RobertR

Senior HTF Member
Joined
Dec 19, 1998
Messages
10,675
Here's some food for thought (pun intended): Most people seem to have a preference between Oreo's and Hydrox, and (among the peanut butter lovers) between Skippy and Jif. To my knowledge, there is no raging debate as to whether discernable differences between these products actually exist, and no one seems to have proposed that any such difference is just a massive marketing ploy. Yet, I wonder how many people could distingish correctly 24 out of 24 times. My guess is that one would get confused on at least some of the trials, and make a mistake. (Let's assume one could efectively wash one's mouth out betwen trials.)
Interesting analogy, Larry. I think it points out the difference between how cookies, peanut butter, and high end stereo components are marketed.
Imagine some company comes out with a new “gourmet chocolate wafer-with-filling cookie”. This cookie is claimed to have all sorts of fantabulous taste qualities, qualities that are said to set it FAR above a mere Oreo or Hydrox. The magazine ads for this cookie are a model of glossy beckoning, clearly associating its buyers with good taste and refinement. There are even magazines devoted to describing cookie differences, and the writers for those magazines are very good at writing page after page of glorious prose, describing the subjective differences between cookies. Naturally, such a glorious tasting cookie doesn’t come cheap. You want the best, you gotta pay for the best, well above what you pay for an “ordinary” cookie.
But there’s a problem. Someone has gone to the trouble of chemically analyzing these gourmet cookies, and found very small differences between them and “ordinary” cookies. Furthermore, double blind taste tests are conducted, and people can’t reliably distinguish them from an Oreo. The gourmet cookie writers immediately denounce these results, claiming that “mere science” can’t explain the taste of a cookie, science is sometimes wrong, tasting cookies under test condition is too “stressful”, tasting the difference takes a long time, etc.
Now the analogy is more accurate :)
 

Geoff L

Screenwriter
Joined
Dec 9, 2000
Messages
1,693
Real Name
Geoff
After reading the 4 pages of input I was prepared to begin to write.
Then Roberts cookie analogy hits me, hard...
It all goes out the window and laughter begins to poor out of me to point of spilling my fresh ground & made coffee, of which I can taste the difference 9 out of 10 times buy the way. ;)
No pun intented to the serious nature of this discussion...
Geoff
 

Larry B

Screenwriter
Joined
Nov 8, 2001
Messages
1,067
Robert (and others):

I am going on the assumption that you, like most of us, have food preferences, based on a life time of tasting various foods. Do you think there's even a remote chance that in DBT you would fail to reliably choose your favorite amongst other related foods? I suspect such is the case. If this occurred, would you happily conclude that your preference was all a myth?

Larry
 

RobertR

Senior HTF Member
Joined
Dec 19, 1998
Messages
10,675
If this occurred, would you happily conclude that your preference was all a myth?
Larry:

I would happily conclude that my perception of the taste was influenced by other things besides what's actually in the food.

Some years ago, there was a saying that went "real men don't eat quiche". There is no doubt in my mind that the publicity surrounding that phrase influenced how some men perceived its taste.
 

Lin Park

Second Unit
Joined
Jan 31, 1999
Messages
286
Here's another question about "The Test":

If the guy has $10k on the line, who grades the test and actually determines how many times I get it right? Can I bring someone with me to stand behind the curtain to verify that he isn't playing the same thing twice and getting me to guess wrong?

The reason this guy wouldn't accept coming to my house is that I'm not gonna make any tweaks to the transport or preamp between tests (which you can't do in 95% of the systems out there). In other words, you'd all hear the same differences I do between my old solid state amp and my newer tube amps.

I agree with all of the things mentioned with regard to statistics here but I still think that guessing at it for 6 tries will ever cause this guy to part with his $10k. If he honestly believes you can't tell the difference then 6 tests should be more than enough (especially since he is controlling the test).

Lin

P.S. Hard to believe this has caused such an uproar - I may have to visit his studio and try this when I'm down that way. If I visit him and use his equipment, I get to pick everything and lose nothing right?
 

Chu Gai

Senior HTF Member
Joined
Jun 29, 2001
Messages
7,270
Well then I won't admit to having a decent recipe for quiche then ;)
BTW, scientifically, those food comparisions would likely have involved triangle tests at some points. Larry, got anybody over there that can crank out the probability numbers (assuming p=0.5) for getting 100% correct answers in 12 trials?
 

Larry B

Screenwriter
Joined
Nov 8, 2001
Messages
1,067
Chu:

Larry, got anybody over there that can crank out the probability numbers (assuming p=0.5) for getting 100% correct answers in 12 trials?
I'm not sure you asked exactly the right question. The probability of getting 12 in a row (if I haven't forgotten the basics) is 1/(2^12). IOW, pretty darn unlikely!

As for p
 

Jonathan M

Second Unit
Joined
Jul 23, 2002
Messages
267
If we set aside the "all amps sound the same" argument and focus on what I think is his real intention, debunking the exaggeration and hyperbole that is everywhere in high-end audio marketing and reviews, I'm pretty much in complete agreement with him.
Well said.
In regards to the peanut butter analogy, I'm sure there'd be plenty of tests if one brand of peanut butter started selling for 40 times the price of another!
Yes, it all comes down to the MONEY involved. This is why you hear such outrageous claims of superiority in both marketing and magazine reviews - how else are they going to sell things for so much money!!
I personally don't believe the sonic differences (if they exist) in the majority of well designed audio gear are significant - Loudspeakers excluded. They certainly are not significant enough for me to spend vast amounts of money on, anyway. Ofcourse, if I had money to burn, I'd certainly purchase a higher costing product - to get the looks, not the sound!
Anyway, in case you're wondering, here's the statistics:
Assuming the appropriate statistical assumptions are met (For a binomial test - ie. independence of trials etc. - which in this case is probably OK since as DIFFERENCES are all that is being observed), the score needed to be obtained to be statistically significant is given (approximately) by:
x = z_p^2*24/(z_p^2+1)
THIS FORMULA IS INCORRECT - SEE MY POST BELOW...
where z_p is the z-score of the confidence level of the test.
With a 95% confidence level, z_p = 1.96, thus one would need to get 17/24 or better.
If one got 23/24 then you could be more than 99.99999999763493% certain that the result was due to more than chance.
 

RobertR

Senior HTF Member
Joined
Dec 19, 1998
Messages
10,675
If he honestly believes you can't tell the difference then 6 tests should be more than enough (especially since he is controlling the test).
So Lin, would you be willing to risk a 1 in 64 chance of having to shell out 10,000 bucks?


Edit for Chu: Said in best Clint Eastwood voice:

"You've gotta ask yourself one question: Do I feel lucky? Well, do you, Lin?"
 

GordonL

Supporting Actor
Joined
Feb 14, 2000
Messages
771
Can I bring someone with me to stand behind the curtain to verify that he isn't playing the same thing twice and getting me to guess wrong?
Seems to me that if you could reliably detect any audible differences, you wouldn't need to do that and hence wouldn't really be guessing. :)
 

Andre D

Stunt Coordinator
Joined
Oct 5, 2001
Messages
70
Just out of curiosity, what type of home setup does this fella have who's making the challenge?
From what I can recall, he has McIntosh amps.

I know 24 tries seems like a lot, but remember most people claim they can clearly hear a difference between amps. All he is doing here is eliminating the variables.

All of the questions you guys are bringing up has been answered numerous times already on Carsound by Richard Clark himself. Even if you are not into car audio you should check it out.
 

John Kotches

Senior HTF Member
Joined
Mar 14, 2000
Messages
2,635
Jonathon,

How does one derive z_p though -- that seems to be missing from the equation.

Based on the calculation results you posted the likelihood of 23 of 24 is roughly in the range of one in 700 million.

edit
I found a web page with a program that does the calculation.... Assuming the web page is correct, it comes out to something like this:
99% confidence (1 in 100) 20.85
99.9% confidence (1 in 1,000) 21.97
99.99% confidence (1 in 10,000) 22.51
99.999% confidence? (1 in 100,000) 22.83
99.9999% confidence? (1 in 1,000,000)? 23.04

Now, I redid the calculation with 240 iterations (10 people), and the numbers changed by a factor of 10.

Should the formula for 240 iterations be "the same" ie

x = z_p^2*240/(z_p^2+1)

I would have expected the number of correct answers to decrease (per centage wise) as # of iterations increased.


Regards,
 

Lee Scoggins

Senior HTF Member
Joined
Aug 30, 2001
Messages
6,395
Location
Atlanta, Georgia
Real Name
Lee
Also, usage of Krell and Denon amplifiers are both excluded.
John Kotches said this on page 1, so that is where I wondered if Krell amps were excluded.
Something really important is missing here:
The Design and Implementation of the Playback System
The only true fair test here given the shortness of the music sampling times is to have a very revealing, detailed playback system.
(Robert, I make no representation for any Stereophile reviewers that hear something on the first note which I would not agree with.)
How do we know Richard Clark is serious is a fair question. That is why I want to see evidence he has actually set aside $10K in escrow. Maybe I will take the test if I can bring along my own playback system.
My preference would be to have very detailed tube amps like Audio Research and an ARC Ref 2 preamp playing over Avalon Eidolons or Maggie 20.1s. It may be interesting to listen to a headphone setup like Stax Omegas. The thin diaphragms allow one to listen to very minute differences in sonics. We use them at the McCoy Tyner session and could hear unbelievably small things.
These are very expensive electronics but we are trying to prove scientifically that sonics are different in complex DBT conditions. The quality of playback that allows minute differences to be heard would level the playing field.
I am also curious why Richard Clark using his EE to work on car sound systems? It seems he would find more challenging work in high end audio. I don't question his credentials, just have a curiosity about his interest in car audio.
Also, I think it is important for Mr. Clark to remove himself from the judging as his money is on the line. There needs to be an impartial referee not connected to either Mr. Clark or the audiophile community.
:)
 

Lee Scoggins

Senior HTF Member
Joined
Aug 30, 2001
Messages
6,395
Location
Atlanta, Georgia
Real Name
Lee
One more (or three) important question:

Who has taken the test before and what did they think about the level of fairness?

Were there any industry types involved?

Were there any high end brands represented?

Let me know.
 

John Kotches

Senior HTF Member
Joined
Mar 14, 2000
Messages
2,635
Lee,
I read the series of posts, wherein it said this
Emphasis mine.
Although some have stated this has been opened to home power amplifiers, it is not explicitly stated in the rules.
Further, if one of the amps dies because it can't handle the load, the test is scrapped. This would mean even if you had answered 11 of 11 correctly to that point, and one of the amplifiers died out on the 12th test, the results would not be counted.
Aggregate results have not yet been made available of # of individuals tested and their results.
Regards,
 

Users who are viewing this thread

Forum statistics

Threads
356,812
Messages
5,123,607
Members
144,184
Latest member
H-508
Recent bookmarks
1
Top