Bob Whitehill, Stereoscopic Supervisor, Pixar : Good morning everybody. I would like to welcome you to the Burbank chapter of the 3D fans' anonymous group.
It feels like we need a support group for those of us who believe in 3D creatively, and that it actually adds more to movies and make them better. Before we dive into how we do 3D at Pixar, I thought I’d talk a little bit about this odd title that I have, Stereoscopic Supervisor. It sounds like some horrible medical procedure. But you know, stereo was not originally an audio term that we're so used to from the '50s and '60s. It was coined way back in the 1850s by Charles Wheatstone. Soon after the invention of photography, he came up with the idea. Well, let’s take a photograph, one for the left eye and one for the right eye. And we can create stereograms.
Stereo is a Greek word that means solid. And so for decades millions of stereograms were taken all around the world and enabled people to feel that they had traveled or seen different shots from the civil war that were very meaningful, San Francisco earthquake, or again something just fine and cool like Mark Twain playing pool [shows those images]. What actually caused the stereograms to sort of go out of favor was motion pictures. So in the 1920s motion pictures caught on. Those seemed a more solid representation of real life than stereograms did, and so stereograms faded away. Now we have this amazing opportunity with today's technology to combine the two. So I think it's a really exciting time. And so it’s sometimes hard to feel that people don’t embrace it and recognize 3D's capability as a creative tool. So I’m going to talk a little bit about Brave, then I'll show a couple of Brave clips and also about Finding Nemo.
It’s not always easy being a 3D geek. So nothing speaks better about our 3D process than the footage itself. So we’re going to dive right in and take a little look at some sequences from Brave. This is one of the opening sequences. Brave was really a terrific show, a wonderful show to work on in 3D because of the wonderful organic environments, you know, a lot of work went into the vegetation. The trees, and the moss, and the hanging vines, it’s really a breathtaking show in just these set designs. That really works well in 3D. Go ahead and don your 3D glasses and we'll dive in. We'll take a little look at one of the opening scenes from Brave. [Shows clip of Princess Merida “taking a day off from being a princess” near the beginning of the film]
So now that we are going back to our 2D slides, we can take the glasses off. So when we first embarked on this new era of 3D about four and a half years ago, it was important that we have a mission statement. We have a grand goal for all the films in the studio. So we talked about a mission statement involving the three Cs, the first one being comfortable. Above all else you know, we felt so good about our movies and how hard we worked on them so we wanted it to be a comfortable viewing experience. We didn’t want to anyone to be nausea, or have a headache. That seemed to be the worst case scenario. So one of the things that we studied in order to make it a comfortable experience is what sort of separations should we strive for. So we studied, interocular distance, the distance between our eyes. And of course they're different for children and women and men.
We like to think about our screen size. We're going to looking at a variety of different screen sizes even up to IMAX which can be over a 100 feet. Now, with home video of course we go down to 40 inches. And so that was another variable we had to take in consideration. The one constant we knew was that we were always going to render at a certain pixel width at 1920. And so in conjunction with working with some of the vision scientists at UC Berkeley, we came up with a basic plan of just measuring these separations in terms of pixels. So what is the difference of the same object in the left-eye view and the right-eye view? And that gave us sort of a short hand to be able to talk and evaluate different shots.
So in a shot like this, [shows 2D image from Finding Nemo] say our primary point of interest would be with Dori or Marlin. We were trying to keep that point at around zero pixels of separation which means it’s at screen depth, you'll perceive it to be on screen just like you're watching a 2D movie where it’s very easy to view. And then we're able to spread foreground objects forward, midground objects a little bit further back and our most distant objects a little further back. So that sort of, you know, just gave us sort of nomenclature to latch onto.
One thing that's going on physiologically when you watch a 3D movie is that you have two main muscle systems in your eyes. One is for focusing on distance. So you're focusing at a distance from you and then your eyes are also converging obviously at that distance. And so our entire life those two muscle systems work in sync. By looking right here at my finger [holds up index finger], here it's focusing at this distance and converging at that distance. And vice versa if I look at it a long distance. What happens during a 3D movie is that changes a little bit because you're going to be focused at that same 3 feet away from you whatever your distance to the screen is. But now you're converging your eyes in and out as those objects move in front and back of the screen. That's why we try to keep our point of interest right around screen. That's the most comfortable place to view. You can vary off of that for short periods of time a lot or for a long period of time a little, but you can't have a layout screen forward or back for a long period of time. That’s when you get eye fatigue headache, nausea, et cetera. We've refined our approach a little bit so that we feel more comfortable with the point of interest being in front of screen, but gracefully in front of screen not “over the top” in front of the screen.
And so we had to make the decisions here for instance [about where things should be in frame]. How far in front of screen should that be? If we're not careful something can suddenly look like elastic, when it comes to going too far out. If it’s too little, it’s not rewarding and since people spend an extra few dollars to see 3D, so we want them to have a rewarding experience. So those are what we call parameters. So one of the first things we strive for to make it a comfortable viewing is smart 3D parameters.
Another thing we look at are called retinal rivalries. And here is a shot from Up. And there’s really no rendering mistake or anything, but you can have a reflection like this in the sword hilt that simply picks up a window in the blimp that isn't seen in the other eye and view in the exact same way. And what’s interesting, if you walk through life and you look at glassy surfaces with the shades that you're getting in each eye are really quite different. But in real life we can handle it. The brain is used to it. But if you're watching it on a screen and focusing at a same distance it can be very jarring. It creates sort of a shimmer effect.
So if you guys are watching 3D and you see something that’s sort of shimmery you can pause it and just toggle your eyes back and forth like that, you'll see that something -- you're seeing something with one eye that you are not seeing with the other, either a different shape or the very object is not in both eyes. So we carefully comb through every frame in the film and look for these retinal rivalries or eye diffs as we call them, and we try to make sure that everything is same between the eyes. Another thing we think about is depth of field. So you know filmmakers since the invention of film have used different techniques to denote depth in a 2D image. So something like Ken’s glitter tux pants which are fantastic by the way [shows sever images of Ken using different depth of field] are very much out of focus and that 2D visual in sense of depth between him and Barbie, but in 3D it could be very awkward and distracting because if you choose to look at them, like I said before you're used to converging and focusing at the same distance, but you're unable to bring them in focus. So we can actually draw your eye to them which is the exact opposite the filmmakers intent. So we make subtle changes to adjust the 3D version of the movie where you might, as we'd say, broaden the depth of field or make them a little less blurry. There are still blurry so that your focus goes to Barbie, but less so. That way they don’t become distracting. And that's the whole thing. Depth of field is also a visual cue for scale. I’m sure you've seen some of these tilt shift photographs and how they tend to miniaturize things and so we want to make sure that we're not shrinking our scenes now by having too much depth of field in 3D.
Another thing we think about our foreground objects. So here is an example from Brave, so here Fergus having his very heart healthy lunch [Fergus had a plate loaded with food and is holding a turkey leg].
Can you see this turkey leg here in foreground? Well, if I left that in the scene then we have to account for that 3D depth, we can't have that so far on the audience space that it’s hitting the guys in the background in the face. But in order to account for it that means that Fergus has to be recessed behind the screen, kind of flattened out. So it makes it a very sort of unrewarding shot in 3D. So for the 3D only version of our movies we will often times take out an object like that, so now I can spread the depth more comfortably across Fergus and have to account less for an immediate foreground object. So those are some of the things we think about when we're striving for a comfortable movie, you know. Every filmmaker should look at least those things to make it a comfortable viewing experience.
And the second C is consistency. If you go through our films there is an amazing amount of variety in terms of color saturation or lens choice or framing. For instance, here in The Incredibles where his life is an insurance salesman that the color is just drained out of the movie, it’s a pretty long lens, it feels very cramped and crowded as opposed to his life as the superhero with all kinds of space and very saturated colors. So in an instance like that if we needed to do The Incredibles in 3D of course we want the first shot to feel very collapsed and this shot to feel very deep. This is a shot from Finding Nemo and as we go through and I play some of these clips you have to forgive the compression quality these are just playing through keynote. Here is a shot of Marlin and Coral at the reef and it’s a beautiful day and it's so excited to be there that we make a nice and rewarding -- but then after the Barracuda come it’s essentially the same lens length, the same camera position, but it’s a very sad moment, so we sort of collapsed this space down to capture the emotional tone of the scene.
Up is a terrific example. Well, here is Carl after he has lost Ellie. The colors drained out of the movie. He feels very cramped and claustrophobic in the composition. In fact for about 50, 55 shots in a row they use the exact same 40mm lens, I mean, it’s a tremendous amount of visual structure that just collapse the feeling of this movie down along with the emotion and the acting. You contrast that with a shot like this later in the movie with deeply saturating colors, deep sets and a lot of more 3D depth. We even attempt to do this with -- in certain sequences. Here after Carl has reached Paradise Falls. It’s not -- the goal is not what he hoped it would be. The color is very flat, the lens is very long, the composition is very static. He of course -- his emotion itself is very static. But as he has his emotional waking looking through the adventure book, the life that Ellie and he shared together, the colors become more and more saturated, the camera angle becomes pan, the lens becomes wider. His expression of course becomes more animated. The FX guys added tears to his eyes and we as a 3D team added more depth each successive time when we turned to Carl it got a little deeper, then a little deeper and a little deeper. So that's what I mean when we talk about consistency we want to enhance and mirror what our counterparts in other departments are doing. Finally if we've done our job right it should be very captivating experience. People are going out of their way to see our movies in 3D and want to grab them. We and want to give them something to think about and be more powerful than in 2D. So we're taking shots like this from Finding Nemo, we don’t want to leave any 3D on the table so to speak. We want to make it a powerful, rich experience.
Both that shot and this shot we call z-axis shot where the camera is flying down the z-axis. It’s a wonderful in 3D to have the environment moving past you like that. We also in Brave had the opportunity for these great helicopter shots where we were able to soar over the environment to really soak it in. The challenge with these is miniaturization, you know, once you are after about 50 yards or so we're all essentially a Cyclopes anyway, there's no longer any variation between what our left and right eyes sees the further you get away from an object. So for a shot like this there is a danger of once you pass that initial turret tower everything is just going to feel flat, it’s going to feel like a 2D movie.
So we have to be very careful about spreading out the cameras and then taking that inter-axial distance farther and farther to get some sense of depth. And not so far that feels like you are looking at a miniaturized set. So that's something we spent a lot of time on. We tried to find that perfect balance between pulling depth out of a shot but not making it feel like a lego set. And of course you want to take advantage of fun shots like this particularly when they're in the middle of a montage about the fair games. But then in a shot like this where it’s a more of a character shot we wouldn’t want to pull the riding toy or the aerials on the side of frame too far forward because then it might distract the audience, people might look to those instead of the character.
And so again this is probably the worst of the compression artifacts. I want to during this shot and show you actually the scene, it goes -- what's so fun about this is that Merida spends a good deal of it in the dark down in the throne room of the castle. And when the background is dark then we don’t have to account for it, you know, in 3D settings so we can dedicate more depth to the foreground character. We're very conscious of -- even if you are looking at Merida, but there is the castle behind her, we don’t want the castle to get too diverse, too blown out because then your eyes has to go really walleyed and it may be painful if we put things back there. But if it’s just dark in the background you can really pump up the stereo primers on the character itself.
All right, guys, let’s go ahead and roll the next brief clip [Merida falls into abandoned throne room]. See if you don’t feel this is more powerful for being in 3D. I have two sons, 8 and 10. And my 10 year old is a little bit more easily scared than my 8 year old and when that scene stated he said “bathroom break’ as soon as he heard the music he was up and out.
All right, so with Finding Nemo – what an incredible opportunity we had with Finding Nemo -- you know, it's not a conversion; it's a recreation we like to call it. You know, the original Finding Nemo was released in 2003, so it was a decade ago and it was fully in production. Now, imagine a live action film 10 years later trying to recreate, to resemble the cast, no one has lot any hair, no one has gained any weight, resemble the set that look exactly the same, lighting just the same, give exactly the same performances and now this time they're going to film it in 3D, well, that's the opportunity we had in Finding Nemo.
This is an example of a 3D Depth Map created by Robert Neuman, the 3D Stereographer on the film. Positive numbers refer to the amount of pixels the image will come out of the screen and negative numbers refer to the amount of pixels the image will go deeper into the screen, creating the 3D depth.
Grey Scale – The final image in the computer representation of depth. Darker images will be furthest away, and lighter images will be closer to the viewer.]
One of the great challenges and one of the great rewards of Finding Nemo is all this floating particulate matter. You know, they put all this flowing particular matter into the original movie to make sure that it felt like the ocean, it didn't feel like a clean aquarium. And so we would have to carefully dial in each one those little particular pieces because if we set the 3D for Bruce the character here, for instance, a lot of that particulate would be too far out into the audience space and so we may kind of crush it, you cull it, you move it and scale it so they're going to work on shot-by-shot basis. And then in a shot like this one of those great Z-axis shots that we were talking about, we actually developed a system that as the particulate specs came closer to camera, they would gradually just dissolve off so that you could still have this feeling of particulate rushing toward us without having to really flattened out the shot, but we didn't want those particulate pieces to just come 80, you know, 90 pixels separated in front of screen, we would actually have them just start to dissolve off once they got to, I think, 20 pixels they must start to dissolve maybe they would be dissolved by 40 pixels converged. You might have gone with 36 pixels, I went with 40 pixels.
And yeah, it's just more of the same -- this was a fun shot to convert because all of those mines are actually a painting, they were all flat in space and so we had to go in and rotoscope out each of those minds and make them place between the space. All right, so guys why don't we gear up that sequence from Nemo and, yeah, hope you enjoy this. Put on your glasses, we'll -- you'll see what I mean by the particulate being both a real boom to volume and space, but also kind of be a problem, so I hope you make good choices that how much particulate to leave in and where to place it [Plays 3D clip from Finding Nemo of exploding mine scene].
See, not a conversion, an actual recreation, we would go back and re-film it. In fact, we re-filmed it at a higher resolution, it was originally 1600 X 900, we rendered at 2K which was many more pixels per frame, which over the course of the movie was 91 trillion something more pixels which should give you a sharper, better picture. So thanks for being part of the support group this morning.