Jump to content



Sign up for a free account to remove the pop-up ads

Signing up for an account is fast and free. As a member you can join in the conversation, enter contests and remove the pop-up ads that guests get. Click here to create your free account.

Photo
- - - - -

Review Scores: Pat and Rich


This topic has been archived. This means that you cannot reply to this topic.
7 replies to this topic

#1 of 8 OFFLINE   Cinema Squid

Cinema Squid

    Auditioning



  • 10 posts
  • Join Date: Sep 02 2008

Posted September 04 2008 - 03:08 AM

Is there a reason the reviews by Richard Gallagher and Pat Wahlquist on this forum do not have content/video/audio/extras scores assigned to them while those from other reviewers (Sam, Matt, Cameron, Neil, Michael, Gary, Todd, Kevin, etc.) do? I do not mean to provoke controversy or imply anything with this question, just curious.

Even though the text of the review itself is far more informative and important, I find that scores are nice way to mentally frame a review in comparison to the writer's own opinion on other titles as well as in the context of other reviewer's opinions on the same title.

I understand that scoring can be a tricky and controversial subject for each category - e.g. content scores possibly needing genre weighting, video scores traversing the minefield of filmmaker intent vs. "demo scale", audio scores with a similar content/intent problem (dialog-driven vs. SFX-laden), weighing the quality vs. quantity of extras, etc. At the very least, I think assigning an overall score would be helpful.

Perhaps I answered my own question in the above, but a discussion of scoring systems in general and the opinions of members of this forum on the subject could also be interesting to have - although that kind of thread does admittedly have a danger of devolving into harsh criticism of individuals.

#2 of 8 OFFLINE   Michael Reuben

Michael Reuben

    Studio Mogul



  • 21,769 posts
  • Join Date: Feb 12 1998

Posted September 04 2008 - 05:24 AM

I won't try to speak for Pat or Rich, but I can tell you that I don't believe in scores, don't refer to them and, on the occasions when I do "guest" reviews (of which I'm currently working on a pair), don't provide them.

I find scoring arbitrary and largely meaningless. At best, it may let you rank reviews by a single reviewer. But since a "B" or a "4" (or whatever) probably doesn't mean the same thing to Reviewer Smith as it does to Reviewer Jones, I find scoring to be virtually useless in comparing different reviewers' outputs.

If I want to know what they think, I read what the reviewers have to say. Allowing for my obvious partiality, I must say that HTF's reviewers don't disappoint. Posted Image

M.
COMPLETE list of my disc reviews.       HTF Rules / 200920102011 Film Lists

#3 of 8 OFFLINE   Cameron Yee

Cameron Yee

    Executive Producer



  • 10,596 posts
  • Join Date: May 09 2002
  • Real Name:Cameron Yee
  • LocationSince 2006

Posted September 04 2008 - 05:27 AM

I think there might be one or two others who don't use scores.

Your query is a legitimate one and scoring has proved problematic over the years for the reasons you and Michael mentioned. I have commented on the subject at least a couple times in my review threads and have considered foregoing scoring, but have found it difficult knowing when/how to make that break.

One thing that should be said is that Ron has given us all a lot of latitude in the way we put together our reviews, which is very empowering but brings with it variation that you don't see on other sites. Over time though, I think people get a sense of where that particular reviewer is coming from and the variation becomes less of an issue.

And no worries about harsh criticisms - I have confidence in our membership (or at least our moderators Posted Image).
One thing leads to another at cameronyee.com

#4 of 8 OFFLINE   Matt Hough

Matt Hough

    Executive Producer



  • 11,477 posts
  • Join Date: Apr 24 2006
  • LocationCharlotte, NC

Posted September 04 2008 - 08:56 AM

A little story from my own personal experience. When I began writing movie reviews for a newspaper syndicate many years ago, I used no rating system at all. After about a year, my editor called me into his office and asked why I didn't use stars or grades or numbers to rate my opinions of the movies. I explained that I wanted the reviews to speak for themselves and that I felt a star system would not help make things clearer. He respectfully disagreed (much to my embarrassment), and I became obligated to use a star rating system.

I've been doing it for so long now that once I began writing here, I just naturally kept on doing ratings. But if it was decreed that we weren't going to do it any longer, I could certainly stop.

#5 of 8 OFFLINE   Cinema Squid

Cinema Squid

    Auditioning



  • 10 posts
  • Join Date: Sep 02 2008

Posted September 04 2008 - 05:03 PM

Thank you for the responses thus far. When I am self-reflective, I agree that assigning numerical scores to art and even mere entertainment is essentially a foolishly unfair practice to the creators of original works. At the same time, the volume of available selection (tyranny of choice) is so great that something must be provided to separate the wheat from the chaff in how to spend one's leisure time (at least for home video in general and even the current Blu-ray release schedule is passing one's own ability to watch *everything*). While deeply flawed, scores are still one of the strongest tools for discrimination in measuring potentially compatible aesthetics between individuals.

The other noteworthy aspect of scores is their ability to be easily collected and processed by the slowly growing semantic web effort - i.e. mechanically gleaning some kind of overall meaning from the vast array of internet content to present in a more queryable and summarized form. One of these small efforts is the hReview microformat and although it does indeed leave ratings as an optional field, there is certainly a case for the more information the merrier.

As a sidenote, one of the interesting things that I have found while collecting Blu-ray review links for my website is that while there can be great variance on scores for single titles and not all scoring systems are completely compatible, there is still a surprising amount of adherence to overall averages when all the scores are taken together as a whole. In general, I have found that most reviewers fall between a 5-to-15 point mean difference (both absolute and average) in scoring in video, audio and extras for Blu-ray discs after scaling to a 100 point range. In other words, most reviewers agree within one-quarter to three-quarters of a star out of five stars on the average title. When taking into account that the granularity of ratings is usually either a half-star (10-point scale) or whole star (5-point scale), there is more reviewer consensus out there than most would guess.

#6 of 8 OFFLINE   Michael Reuben

Michael Reuben

    Studio Mogul



  • 21,769 posts
  • Join Date: Feb 12 1998

Posted September 04 2008 - 05:13 PM

Quote:
At the same time, the volume of available selection (tyranny of choice) is so great that something must be provided to separate the wheat from the chaff in how to spend one's leisure time (at least for home video in general and even the current Blu-ray release schedule is passing one's own ability to watch *everything*).
That one's easy for me, because I start from a simple premise: Is the content (film or TV show) something I'm interested in? Since I'm usually up to date on what's out there, that narrows it down to a manageable number. Then it's just a matter of making sure the technical quality is acceptable.

M.
COMPLETE list of my disc reviews.       HTF Rules / 200920102011 Film Lists

#7 of 8 ONLINE   Ronald Epstein

Ronald Epstein

    Studio Mogul



  • 40,282 posts
  • Join Date: Jul 03 1997

Posted September 05 2008 - 11:45 PM

As has been posted, we give our reviewers quite a bit of freedom
in the manner of which they review product on our forum. This is
the reason why each of them have their own individual style.

We are currently working on a template that will be completed
in a few months. This will not only bring uniformity to our reviews,
but will act as a huge database where you can search for product
based on a wide range of choices (genre, actor, rating, etc.). There
will also be a star rating system for the film, video and audio.

 

Ronald J Epstein
Home Theater Forum co-owner

 

 Click Here for the latest/hottest Blu-ray Preorders  Click Here for our complete Blu-ray review archive

 Click Here for our complete 3D Blu-ray review archive Click Here for our complete DVD review archive

 Click Here for Blu-Ray Preorder Release Schedule  Click Here for forum posting rules and regulations


#8 of 8 OFFLINE   TonyD

TonyD

    Executive Producer



  • 16,203 posts
  • Join Date: Dec 01 1999
  • Real Name:Tony D.
  • LocationDisney World and Universal Florida

Posted September 06 2008 - 02:37 AM

i look at the star rating of the film, but for the rest of the disc i just read the opinion of the reviewer not the stars.
facebook.com/whotony