Review Scores

Also, to put it into perspective, we don't review every single piece of gear out there- this would be impossible. We are probably guilty of cherry-picking gear that we suspect will be good or perhaps outperform the competition- I know that I certainly push harder to review pieces that I think are worthwhile. This creates a tendency towards higher scores.

Could either of you guys respond to my "Bluejeans cable review" thread?

thanks

Mika, kindly as a matter of Forum decorum don't bring attention to another thread if it's not related to this one. Thank you.

Seeing as this thread was about DVD Etc. reviewing procedure, I thought it was related.

Not to belabor it, but this thread pertains to the scoring system the editors use to rate reviewed products instead of the testing methodology used in assessing cables and interconnects.

Clint et al, care to check out Mika's post? (So there!)

Hello DVDetc,
I wondered why your magazine seems to like everything! When I saw the first issue of your mag, I thought you were some kind of industry publication; and "advertorial" if you will.

My personal opinion: I often learn more from negative or mixed reviews that positive ones. I think you are doing your readers a disservice by publishing reviews of only the best equipment. Your readers are also looking to you to help them avoid high-profile or heavily promoted gear that doesn't measure up. You can't help your readers separate the wheat from the chaff if you only mention the wheat.

I think DVDetc would be a much more interesting magazine if you simply selected what you wish to review - based on new technology, profile/availability, feature set, etc - and then published your conclusions, good, bad, and ugly. To do anything else is to risk the impression that you're afraid to call 'em as you see 'em because you might risk advertising revenue.

Oh, and please find a proofreader. I'm sure this is just growing pains, but I've never seen a national publication with as many typos as DVDetc.

Best of luck,
Bob

Hi Bob,

First as this pertains to typos the title of the magazine is DVD ETC. Magazine not DVDetc. Being partial to perfection I thought you may want to know this for future reference.

As timing would have it, we did just replace a copy editor. However, I will point out that with two magazines in the stable, there are over 1.6 million words to be read in a year between two people. A typo is something we will always face be it a probablity of chaos-factor or an acceptable percentage--margin of error.

Getting to your idea that we "seem to like everything" makes me wonder if you missed reading reviews such as those of the Pioneer and Apex, McIntosh and others. Regardless, I don't beleive that people buy magazines to read negative reviews. In fact, if a product fails to meet a minimum criteria, then I am more prone to pull it--why waste the paper?

Perhaps you do not understand our scoring system? We have run many reviews of products that scored "below average" (90).

Interesting how you are not arguing our specific findings on a given product, but rather asking why we don't pick them apart more. Actual performance is more black & white where as nit-picking the remote, menu, finish, etc. is more a matter of opinion. We do not call in products that we know are defunct and likewise, manufacturers will not send us products that they know are defunct.

I hope this adds some understanding to your query.

-Clint

I wouldn't say we only review "cream of the crop gear" by any means. We've reviewed everything from $60 Mintek DVD players to Apex televisions, to the higher end gear. Our scoring system is defined in every issue--few take the time to read it or understand it. They are used to other magazines rating systems and don't understand that 90 is the average--anything below 90 is below average--and I personally wouldn't buy anything below average. Some magazines don't even score their reviews!

Hi Trevor, The symbol/color-code is what consumers look for. For example, if they now their budget warrants high-end choices, they will look for reviews of Black Diamond components. Entry-level budgets will look for Green Circles and those Leprecahns will always be after me clovers! (Sorry, I couldn't help myself.) I put 3 hours of work into this scoring system--how can it be flawed?! (I'm kidding of course.) Does my madness make any sense? Anyone?

Clint,
Touche on the name of the magazine! Maybe I only value perfection in others ;-)!

First, I do wish DVD ETC. all the best; it can only be good for everyone to have another healthy mag out there. I fear I was a bit critical. Maybe I can be more constructive (and I've picked up a couple of issues at the newsstand, so I haven't seen all your reviews).

Maybe I am not part of the audience you're after, but I certainly DO read a magazine in part for the negative reviews. It's not a waste of paper for me because I become aware of the pitfalls of poor design and implementation, and know what products to avoid. Manufacturers should be held accountable in the press for releasing shoddy products.

And I would say that in this era where, frankly, most products are at least competetent, buying decisions are often based on what might be considered "nitpicks". I think many buying decisions are based on ergonomics, remote design, menus, etc, the "little things". I would venture that most of the electronics purchases I've regretted have not been because of performance, but because of a lousy user interface or a relatively minor annoyance. I really like a reviewer to "pick apart" the products he or she reviews - it makes me a more informed consumer and a happier post-purchase camper.

There are things I do like very much about your magazine. I like the "let's make sense of this new technology" articles. I like the layout and presentation. Your writing is refreshing and straightforward, and blessedly free of audiodrool rhapsodies and philosophical meanderings of some of your competi...er...peer magazines.

So to sum it up, I'd say do the nitpicking for me, so I don't have to do it myself after buying the product. And never be afraid to be opinionated about what separates the merely competent and good-for-the-money from the outstanding. Vigorously shame the average and praise the best.

So really, all the best! Follow your vision and you'll be fine.
Bob

Bob, You're a gentleman and a scholar. I agree that it is the subtle nuances that separate the average from the "worthy of purchase" in today's manufacturing climate. Sometimes taking the "simple" approach leaves us with little option to calling out technical quirks. Though we address important issues (such as the chroma-bug) and have called out every single player with the defect, where others have overlooked, or chosen to ignore this defect all together.

You may also find it interesting that I did not pull editors already of this industry when we launched this publication. I chose instead to seek out "enthusiasts" with a fresh approach (and non-inflated ego) to review gear. One of our writers, Greg Robinson was even recruited off of the HTF.

Some say I was creating a magazine for the people, by the people. I can't disagree.

I will also tell you that the feedback I have received (from members of this forum) continually help us to become a better publication. I have even mentioned this credit in my Fade In (Ed Note) in the past. We will never be on cruise control--we are always working to best ourselves.

A sincere thank you for your feedback.

My ears were ringing, so I thought I'd come see what Clint was saying about me now. Bob, you make an excellent point, and it's one I often think about when reviewing gear. I can't tell you how often I've returned something, or sold it to unsuspecting ebayers (sorry), because of a miniscule annoyance that grows bigger with every passing day. Whether it be a button which is frustratingly absent from a remote, a noisy tray mechanism, or the inability to back up in an on-screen menu - I promise to do my part in revealing these little quirks that we all consider to be not-so-little. Thanks for your support.