2017 CE Week TV shoot out

I had the pleasure of being one of the judges for the CE Week 2017 TV Shootout in NYC today.  I flew in earlier in the week to be part of the calibration of the 6 UHD HDR 65” TV sets on Tuesday.  The TV Shootout originated from our friend Robert Zohn at Value Electronics.   Robert has been doing this for years previously at his location in Scarsdale, NY.  It recently has switched to being dealer led to being hosted by a neutral party (CE Week) and hosted by Joel Silver (founder of ISF) and Kevin Miller (one of the first certified ISF calibrators).  Unfortunately, we were only able to judge the displays from 9AM to 2PM and I personally didn’t have enough time to go through test patterns and familiar content to spend as much time as I would have liked to, but I did have enough time with enough patterns and reference content that it was easy to pick the first three display units.

The sets that were reviewed varied dramatically in price and in technology.  Two were OLED based (Sony and LG) and the rest were LCD LED based some with local dimming and the rest with edge lighting.  Off axis viewing on the LCD’s was still a huge problem and none of the LCD’s were particular stronger than their competitors with off axis viewing.  Thank god there were no curved displays!

The judging was done across three sections:

  • Studio lighting/ Home heater
  • HDR
  • Streaming performance

In each of the sections there were 4 categories:

  • Perceived contrast ratio (40%)
  • Color saturation and color volume (20%)
  • Colorimetry, Grayscale tracking accuracy, Gamut coverage, Saturation (20%)
  • Motion Artifacts, Perceived lag (20%)

The runner up for all three categories was the Sony XBR-65A1E OLED and the winner by a very thin margin for all three categories was the LG 65E7P OLEDI voted in the same order for the first, second and third displays as the other judges.  The two things I liked about both Sony’s was that both displays did the best job on 1080i content.  The other four displays had a significant artifacts.  So, if your primary viewing is from cable or OTA and that is going directly into your display, you might want to opt for the Sony.  In regards to audio tech, the Sony XBR-65A1E OLED has their Acoustic Surface™ technology which essentially eliminates any speakers on the edges of the TV.  This is like having a transparent screen in your projector based home theater and allows the dimensions of the display to be smaller for the same screen realestate.  While the OLED panel for the Sony is sourced from LG, one would think that the performance would be identical, but they are not.  Sony has different electronics driving the panel and that resulted in different performance.

In the end, I gave the edge to the LG for a couple of reasons.  The first was I felt that the color was a little more accurate than the Sony.  Both were very close in gamma tracking and gray scale accuracy.  Both had a significantly wider off-axis tracking than the LED models, but still required being on axis to get accurate results, particularly with yellow.  While the judging criteria did not include price, the LG is $200 cheaper list price.  If the Acoustic Surface™ technology is important to you, then you probably want to go with the Sony OLED as it is really nice having the sound come right from the screen with no visible speaker on the monitor.

Details:

Calibration started on Tuesday.  The reference meter used was the Minolta CS-2000 spectroradiometer.  Because the meter is so slow it was used to get a reference measurement for each display unit and then the display calibrations were measured with a Kline K10-A colorimeter.  Test patterns were produced from the Murideo SIX-G 4K HDR Pattern Generator, which was driven from the just released CalMAN 2017.  All UHD HDR content was played from the UDP-203 player.

The Sony BVME251 OLED was our reference monitor and was primarily used for the judges to not only look at calibration data but look at reference test patterns and UHD HDR content.  It’s an incredible reference monitor, but at $16,500 it might be out of many folks price range for a 24.5” monitor!  No need to display any test results as they are perfect.  Off-axis viewing on this set is insane.

Below are the displays from lowest price to most expensive with data from the calibrations and my personal impressions.

Westinghouse WA65UFA1001 LED LCD:  The Westinghouse comes in at the lowest price of any of the contestants $650!  A 65” LED for that price is incredible.  It is geared towards streaming content with built in Amazon TV including Alexa.  A full-on Fire TV costs $99, so the price of the TV effectively comes down to $550!  However, you get what you pay for regarding performance.   For instance, there was no CMS and next to zero display controls.  If I had a workout room at my house, this would be the display I would be getting.  65” for essentially $550 to have for non-critical viewing is a pretty good deal.

The unit was so new from the manufacturer that it wouldn’t play UHD content, but once we connected it to the Internet, a FW update came down and the machine played in UHD content.  I also noticed something with the area around the bezel where it first looked like there was some caulking, but when I got close it was clusters of pixels that were dark.  This unit came in at the last minute, so it isn’t clear if this was a pre-production unit or not.

The performance was also the weakest among the units.  It was the dimmest of the displays in UHD mode and color gamut, gray scale and gamma were the weakest of the sets.  Below you can see that the color gamut is a bit of a mess with under saturation for green, cyan, red, yellow and magenta and nonlinear tracking for blue, cyan, green, and magenta.

Gamma was also non-linear.

Gray Scale was also disappointing.

 

Vizio P65-E1 LED LCD: The Vizio comes in at $1699 and provides great value for the price that includes local dimming and Dolby Vision support.  For the extra $1K over the Westinghouse you get a display that has all the proper menu display settings and CMS.  The light output was brighter than the Westinghouse, but not at the same Nit’s level as the Sony’s and LG units.  If you are on a budget and you want something that has good accuracy, then the Vizio is a great candidate.  Sitting on-axis this display was my second pick for LCD displays.  In the streaming section, I did notice motion artifacts that only appeared on the Westinghouse.  However, that was 1.5 screen height viewing distance and most folks are not going to be that close and would not see them.

The Vizio has a decent color gamut and outperforms the higher priced Samsung.

The grayscale tracked quite well compared to the 1st place LG.

Gamma tracked quite well and wasn’t that far off from the Sony and LG.

Samsung QN65Q9F LED LCD:  The Samsung prices in at $3999, over double the cost of the Vizio.  This year Samsung discontinued their local dimming displays, which was unfortunate as both the Sony and Vizio had local dimming.  Not having local dimming for HDR content essentially removed the Samsung from the competition for HDR content.  The artifacts of having an edge lit display are pretty horrendous compared to local dimming.

I felt that the display performed worse with the patterns and HDR reference material that we were looking at compared to the Vizio.  The meters also validated what I was seeing.  Unfortunately, I couldn’t recommend this display to anyone.

The color gamut was less accurate than the Vizio.

The gray scale was very good.

The Gamma was slightly better than the Vizio.

LG 65E7P OLED: The LG OLED placed 1st in the ratings but was the cheaper than both the Sony’s.  It comes in at $4799 and had the most accurate color gamut of any of the displays.  Part of the reason could be that LG has full CMS and ISF modes for calibration where neither of the Sony’s offer CMS.

The LG and the Sony OLED are just incredible to watch.  No artifacts of bright objects on black backgrounds that even the best local diming LCD displays can’t hide.  I haven’t seen inky blacks like this since the Pioneer Kuro display back in 2008.  What you get with these displays is the blacks!  The LG clocked in at 722 Nit’s, with only the Sony LED coming in higher at about 1,200 Nit’s.  However, the perception of more contrast can fool the eyes when there are inky blacks vs. dark gray.  I watch BBC’s Planet Earth II for a couple hours on the displays and couldn’t stay away from the LG and the Sony OLED.  Planet Earth II was authored at 600 Nit’s, so you couldn’t really see a discernible difference from the “put a sunburn on your face” Sony LED and the two OLED’s.

The real kicker for me was still in the blacks.  In the movie “Lucy”  the Sony LED had incredible brightness just after Lucy touches Lucy’s finger and you see the sun come from behind the earth.  The black details in the earth were just lost on the Sony LED and were just delicious on the LG and Sony LED.

If you haven’t seen the LG OLED, you need to go out and see it.  You are not going to see better inky blacks at this point in time than from this LG or the Sony OLED.

The test pattern data validated what I was seeing.

The LG clocked in with the most accurate color gamut.

Very good gray scale, though not as accurate as the Sony.

Gamma was a little disappointing.

Sony XBR-65A1E OLED:  The Sony OLED prices in at $4999 and as mentioned above has the Acoustic Surface™ technology.  While Acoustic Surface™ technology has nothing to do with video display technology, it is nice having a display that doesn’t have speakers decorating the sides or bottom of the display.  If you are going to mount this display on your wall then you need to go to the store and see if you can tell the difference between them with content.  If you are going to have the display on your furniture, then you might be disappointed with Sony’s “kickstand”.  I guess Sony didn’t want to mess with the great aesthetics of the display not having a speaker and didn’t opt for a TV stand.  However, the “kickstand” causes the display to lean back, which I found very distracting and just seemed ridiculous to have it off axis when sitting in front of it.

While the meter shows the LG and the Sony are different, unless you have the reference monitor next to it, you will be hard pressed to notice the difference.  The Sony OLED was a little dimmer than the LG for disc based content about 100 Nit’s less than the LG, but was brighter on streaming content.  The LG and the Sony were brighter than the reference 100 Nit’s for streaming content and the Sony was 10% brighter than the LG.

The data from the meters shows that the color gamut was under saturated for red, magenta and blue and was nonlinear for yellow.

Gray scale was very good.

Gamma was also a little disappointing.

Sony XBR-65Z9D LED LCD:  The LCD Sony priced in as the most expensive display at $5499.  While it has the most light output of any of the displays at over 1,200 Nit’s, it hurt my eyes after extending viewing in a light controlled room.  When we judged the displays with lights on the extra light output became obvious.  If your primary viewing environment is going to be in bright rooms, this display should be one that you consider.

The downsides for this are the artifacts of LED technology.  Even with local dimming, I could still see halo’s around bright objects that were on black backgrounds.  While this was head and shoulders above the Samsung and Westinghouse, it didn’t hold a candle to the LG and Sony OLED which were incredible.  The other downside is the off-axis viewing.  Unless you are on-axis in your room, the picture quality starts to deteriorate rapidly when moving off axis.

For $500 more, I couldn’t see myself buying this over either the LG or Sony OLED.  Yes, it was brighter, but I value seeing real black over getting a sunburn on my face.  I also value being able to be off-axis more with the OLED’s than the LCD’s.  I’m not sure why Sony has this at a price premium over the OLED display, but I would opt for either of the OLED’s.

What I saw on the display matched what the meters were showing.  I thought the color wasn’t as deep as the Sony OLED or the LG.  The color gamut from the Sony proved this out with most of the colors being under saturated.

The gray scale was among the most accurate of all the displays.

The gamma was about as accurate as the other top contenders.

 

 

 

 

 

 

 

 

 

 

Home Theater, Photography, Basketball, Scuba Diving, Mountain Climbing, Racing, Biking, Software development

Post Disclaimer

Some of our content may contain marketing links, which means we will receive a commission for purchases made via those links. In our editorial content, these affiliate links appear automatically, and our editorial teams are not influenced by our affiliate partnerships. We work with several providers (currently Skimlinks and Amazon) to manage our affiliate relationships. You can find out more about their services by visiting their sites.

Share this post:

View thread (19 replies)

DavidMiller

Screenwriter
Joined
Feb 16, 2004
Messages
1,078
Location
Kirkland, Wa
Real Name
David Miller
If they could only make a OLED in the 85" range for the $10,000 or less the "traditional" 4K TVs I would be at the store buying one today. :)
 

daimler2015

Auditioning
Joined
Jul 15, 2017
Messages
1
Real Name
Michael Wheat
Do you guys know if there is a layman's explanation as to the interpretation of the types of graphs and charts shown above?
 

Kevin Collins

Former owner, from the first state with COVID-19
Moderator
Senior HTF Member
Joined
Jan 14, 2007
Messages
4,707
Location
Seattle, WA
Real Name
Kevin Collins
Do you guys know if there is a layman's explanation as to the interpretation of the types of graphs and charts shown above?

Hi Michael,

The saturation sweeps or what I refer to as the color gamut show how well the display is tracking to the primary (RGB) and secondary colors (Cyan, Magenta, Yellow) for each display. They are also five point readings for each color. A black dot represents where the display actually was reading in comparison to where the reading should be. Three things to look for are saturation, accuracy and linear tracking.

Under saturation would be where the black dot is not as far out as the associated square. The Westinghouse shows this flaw for red.

Proper saturation is looking at the outer most squares and seeing the black dot in side the square. The LG is almost perfect.

Accuracy is how close each black dot is to the corresponding square. The LG is the best here and the Westinghouse is the worst.

Linear tracking is how straight the black dots are coming from the center. The Westinghouse has crazy curves for blue and red.

The gray scale is how closely the display is tracking to D65 (Calvin 6500 temperature). This is shown in the sweeps by looking at the center square and the associated white dot. If the white dot is in the square, then it is accurate for the equivalent of a 100 IRE pattern (solid white). The gray scale graphs show a reading for each IRE pattern (10-100 in 5 IRE steps). IRE of 100 is solid white. IRE of 0 is full black. Looking at the gray scale provides much more information about how accurate the color temperature is across all levels of light output on the display. Having a graph that matches the 0 line is what is desired. The Sony LCD was the best of the bunch here and the Westinghouse was the worst. The Westinghouse was accurate at 100 IRE, but all over the place for the other IRE's. Gray scale is most noticeable on B&W movies like Casablanca and the Westinghouse would be showing different color temperatures for the different shades of white (not good).

The gamma is how accurately the display is tracking to brightening or darkening darker areas. If the graph isn't flat then the display will be either brighter or darker depending on where it should be tracking. The biggest offender here was the Westinghouse and the closest was the Samsung. Unfortunately, none of the displays we say had as good of gamma as I have seen on other display systems.

At the end of the day, the data is one thing and what you see is another thing. This is why it is always important to use both. This is why we had a reference Sony BVM reference monitor so we could always look over at what displays are being used in the studio when the title is being color graded. At the end of the day, you want to be seeing what the colorist is seeing in the bay. The colorists are typically using the Sony BVM's and you want your display to match that. Unfortunately, no display does that, yet.
 
Last edited:

Kevin Collins

Former owner, from the first state with COVID-19
Moderator
Senior HTF Member
Joined
Jan 14, 2007
Messages
4,707
Location
Seattle, WA
Real Name
Kevin Collins
If they could only make a OLED in the 85" range for the $10,000 or less the "traditional" 4K TVs I would be at the store buying one today. :)

It will happen.... Yields will eventually improve and the price will drop. The fact that the Sony LCD costs more than the Sony OLED is a head scratcher.... Panasonic is making OLED panels, just not selling them in the US. Supposedly they are excellent displays. Eventually the tech will improve and more manufacturers will be making OLED and then the prices will plummet even more.
 

Sam Posten

Moderator
Premium
HW Reviewer
Senior HTF Member
Joined
Oct 30, 1997
Messages
33,671
Location
Aberdeen, MD & Navesink, NJ
Real Name
Sam Posten
I know. I keep going back and forth between 65" OLED and 75" 940E...

That was me. I went with the 940E, no regrets. My only complaint is on cable shows with fast pans the lighting gets a wonky vertical dark spot. On 4k material and Blurays it is PERFECT. I'll love an OLED when they get to the right price. We got 1/3 of the way there with the early 2017 price drops and 2017 model additions of DV. See you for a 75" OLED in 2022 =)
 

sidburyjr

Second Unit
Joined
Jan 5, 2007
Messages
411
Location
Mount Pleasant, SC
Real Name
Dick Sidbury
Is there a chance that in the 2018 shootout that this year's winner could be included. It would allow us to see how much units have improved.
 

gadgtfreek

Supporting Actor
Joined
Oct 13, 2014
Messages
856
Real Name
Jason
I don't think they do that generally. I always look for techincal posts from the calibrators after the model has been out a little bit to determine what is actually better.

It did not take long for them to report the 2017's had better ABL tendencies, even more accurate color and better tone mapping than the previous year. Also that near black uniformity was improved.

I'd also recommend subbing to the HDTV Test channel on youtube, Vincent Teoh does excellent 15 minute reviews on a display that are highly informative.
 

Kevin Collins

Former owner, from the first state with COVID-19
Moderator
Senior HTF Member
Joined
Jan 14, 2007
Messages
4,707
Location
Seattle, WA
Real Name
Kevin Collins
Is there a chance that in the 2018 shootout that this year's winner could be included. It would allow us to see how much units have improved.

Hi,

That's a great idea, but I don't know if they would be able to get their hands on the same display. Unfortunately consumer displays have a good variance display to display. We were talking between the judges and the calibrators about how displays can be -+ 100 nits for the same display. The same happens in projectors where there are 3 chip designs and they all have a variance of error on registration. I think we had a bright LG based on what others were saying.

The good news is that we have the calibration results (above) and will be able to compare them to next years calibration of the displays and that is the best way to make an apple to apple comparison.

If I go again next year, I would post the same test results with the winner from this year.
 

sidburyjr

Second Unit
Joined
Jan 5, 2007
Messages
411
Location
Mount Pleasant, SC
Real Name
Dick Sidbury
Hi,

That's a great idea, but I don't know if they would be able to get their hands on the same display. Unfortunately consumer displays have a good variance display to display. We were talking between the judges and the calibrators about how displays can be -+ 100 nits for the same display. The same happens in projectors where there are 3 chip designs and they all have a variance of error on registration. I think we had a bright LG based on what others were saying.

The good news is that we have the calibration results (above) and will be able to compare them to next years calibration of the displays and that is the best way to make an apple to apple comparison.

If I go again next year, I would post the same test results with the winner from this year.

I sort of skimmed the data that you gave above (but promise to go back and look at it more carefully RSN) so I'm not sure how much difference there was from set to set in the above evaluation. Is the following statement too liberal an interpretation of the quoted comment. "There's a lot of sample to sample differences between consumer sets even with high quality calibration so that possibly if you had different samples the results might have been different this year." Your original post as to the subjective difference between the various sets caused me to wonder how much difference one would notice without a side by side comparison.

Final questions: How is it decided what brands/models will be evaluated? And who supplies the sets and how do they pick them? That is if they are supplied by the manufacturers do they hand pick the units being evaluated?

Thanks for the original detailed report and the additional answers that you provided.
 

Kevin Collins

Former owner, from the first state with COVID-19
Moderator
Senior HTF Member
Joined
Jan 14, 2007
Messages
4,707
Location
Seattle, WA
Real Name
Kevin Collins
Big announcement on the LG 65" OLED!

The price has been reduced to $3197 at Value Electronics and you can get a free LG Slim wall mount. This is a sizeable discount from where it was when we did the shootout.
 

Kevin Collins

Former owner, from the first state with COVID-19
Moderator
Senior HTF Member
Joined
Jan 14, 2007
Messages
4,707
Location
Seattle, WA
Real Name
Kevin Collins
I sort of skimmed the data that you gave above (but promise to go back and look at it more carefully RSN) so I'm not sure how much difference there was from set to set in the above evaluation. Is the following statement too liberal an interpretation of the quoted comment. "There's a lot of sample to sample differences between consumer sets even with high quality calibration so that possibly if you had different samples the results might have been different this year." Your original post as to the subjective difference between the various sets caused me to wonder how much difference one would notice without a side by side comparison.

Final questions: How is it decided what brands/models will be evaluated? And who supplies the sets and how do they pick them? That is if they are supplied by the manufacturers do they hand pick the units being evaluated?

Thanks for the original detailed report and the additional answers that you provided.

Hi Dick,

It was difficult to see a difference side by side with the LG OLED and the Sony OLED. There was a big difference between both OLED's and the rest of the LED local dimming sets and a huge difference between the all the sets and their LED edge lit competitors. In most cases you pay for what you get. The exception to the rule was the Samsung where unless there is steep discounting, it is not a good value for the money. If you don't care about picture quality and all you want is size, then the Westinghouse is something to look towards. If you want the best bang for your buck with local dimming, then the Vizio is a great choice. If you need super bright and accurate, the the Sony LED is great. If you want the best picture quality and best price for that quality, the LG with its recent price drop is the best deal.

I don't know how the displays were picked, outside that they had to be the latest models. AFAIK, the displays were sourced from manufacturer inventory. CE Week is independent as is ISF, so there are no potential or perceived conflicts of interest with a dealer running the show. The units we had were not hand picked.
 

Robert_Zohn

Value Electronics
Premium
Insider
Joined
Sep 17, 2005
Messages
1,494
Location
Scarsdale, NY
Real Name
Robert Zohn
LG lists my 2017 TV Shootout on their website and credits Value Electronics for our Trademark ownership and as the creator of the event.

497f7e5c85ea54f592b00f7f1a45d068.jpg
 

Edwin-S

Premium
Senior HTF Member
Joined
Aug 20, 2000
Messages
10,000
I really don't get the hatred over a curved display. I had one for awhile and you don't even notice the curve after a short period of viewing.
 

Edwin-S

Premium
Senior HTF Member
Joined
Aug 20, 2000
Messages
10,000
Sony has different electronics driving the panel and that resulted in different performance.

For a short time I was considering the Sony A1 as a replacement for my flawed EG9600, since I was so damn mad at LGs terrible response to the problem. I was watching a comparison of the A1 to a B7 on a British calibrator's website: my current set is the B7. Apparently, the Sony tone maps differently than the LG sets and is set to clip anything above 1000 nits when it comes to bright detail, while the LG will try to resolve bright detail even if the material is mastered out to 4000 nits. He used a piece from a film where (I believe) Alec Baldwin is wearing a bright white dress shirt. The mastering had been done out to 4000 nits and there was a noticeable difference between the two images.

It is interesting that Panasonic's OLED offering was selected as the general favorite during a similar shoot out put on by the website that Vincent (the calibrator I was watching) writes for.
 

Edwin-S

Premium
Senior HTF Member
Joined
Aug 20, 2000
Messages
10,000
After reading this I'm glad I didn't opt for the Q9 as the replacement for my EG9600. The colour volume on Samsung's Q series TVs looks incredible; however, the set lacked in overall PQ, even with the greater colour volume. I'm glad these test results supported my general impression of the these sets from just looking at them.
 

Aaron Silverman

Senior HTF Member
Joined
Jan 22, 1999
Messages
11,411
Location
Florida
Real Name
Aaron Silverman
I don't get the whole kickstand thing with the Sony. Sounds like it's basically a dealbreaker if you aren't mounting the set on the wall.
 
Most Popular