Billy Lynn’s High Frame Rate Walk

In this installment of End Run Chat, we debate the use of HFR (High Frame Rate) in Ang Lee’s new movie/world’s longest camera test. Among other things, it was shot at a staggering 120 frames per second. The transcript below was edited for length, but not for cussing.

Is 48p an “Uncanny Valley”?

Pliny (John Eremic, Endcrawl Co-founder): Our topic is Billy Lynn, the first feature film ever to be shot, finished, and exhibited in 3D, and 4k, and 120 frames per second.

This technology is so new that there are only 5 theaters in the world that can project it properly. There’s a lot to unpack, so let’s jump right in.

Question #1. An industry colleague once told me that 48p is an “uncanny valley” of temporal resolution, but 120p is something different. Something special. Do we agree? Sitting in the theater, is 120p noticeably different from 48p (a la The Hobbit)? And is it special?

Katie (Katie Hinsen, Finishing Artist): I think a lot of people were disappointed with The Hobbit’s “HFR” (High Frame Rate) because it wasn’t high enough. 48 has a noticeable smoothness but anything under 60 really doesn’t have many of the advantages of HFR.

Jamie (Jamie Metzger, DIT to the Stars): I completely agree with the statement that 48 is an “uncanny valley”. It’s not “enough” frames per second to really make a difference, but it is enough FPS to make the audience uncomfortable. We all seem to have an emotional connection with 24FPS; so much so that even 25FPS feels odd to us. I think you really start to feel the benefits of Temporal resolution when you pass 60FPS. The 120FPS in Billy Lynn was really interesting. I got used to it very fast, but what I noticed is that there is no room to hide/no forgiveness. Acting, Makeup, Lighting, sets, Camera moves….all have to be perfect. A director will hate themselves for rushing out of a scene or setup. A director can no longer say “fix it in post”. They will hate that decision with 120FPS. IT IS THE TRUTH.

Katie: Yes. But a lot of that was also due to having the “whole shebang”. The Hobbit’s big downfall was that it was the wrong film for the tech.

Barbie (Barbie Leung, Cinematographer): I did not watch the Hobbit in 48 when it came out, so I can’t speak to that. What I would say of my first reaction is that it felt akin to a PC game cut scene. I am not a gamer, but have watched a fair amount of cut scenes over the last 10 years and have watched that uncanny valley narrow. However, my favorite scenes and sequences from Billy Lynn are where different realities where superimposed… magical/dream moments in the script.

Jamie: I know Douglas Trumbull was testing 144fps, and I would love to see that, but I’m not sure 24 more frames would make a big difference. I would love to have see Billy Lynn in HDR! The Hobbit should have been 60+, 3D, 4K and HDR.

Katie: I disagree with Jamie on The Hobbit, I think that the tech is wrong for “fantasy”. It’s a technique that removes fantasy, and highlights a sense of “reality”. So people saw it and said “hey, this isn’t real, it looks fake”. Where Billy Lynn really worked was in the scenes that played the most “real”.

Pliny: Interesting that you bring up HDR Jamie. One thing that gets overlooked in this whole 3D/4k/120p business is that Billy Lynn was also color graded for 28 foot-lamberts. Which is not the same thing as HDR. But typically, 2D movies are graded for 14 Ft-L, and 3D movies for 3.5 ft-L.

Katie: Yes, they purposely made it brighter than usual. Not HDR, but brighter to make it a better version of 3D. I loved the vibrance and brightness. It took out the issue of 3D being so dim.

Jamie: BL was shot on the F65 so an HDR pass is totally possible. Can’t believe they skipped out on that, but maybe the tech isn’t ready. I thought BL was bright enough, saw it at the Arclight in Hollywood. Nice bright image on the screen.

Pliny: Okay maybe this is where I should just pack up and quit the industry, but I didn’t have the same reaction to 120p as most of you. I didn’t feel like I was seeing something appreciably different from The Hobbit. Granted, that’s four years ago. But my gut reaction to BL was: “Oh. More of that.”

Jamie: Are you saying that you did not “feel the difference” of 120 vs 24 vs 48?

Pliny: 120p vs 48p.

Jamie: I think the football scenes where you can see the camera panning with a football in midair are a great example of temporal resolution benefitting the medium. I saw no strobing at all.

Pliny: So … great for Monday Night Football?

Jamie: Yes. Also think about golf…you’ll be able to see the ball in mid-air!!!

Pliny: I know we’ve all been longing for that.

Katie: Back to what Barbie said about 120, I think it’s important to explore HFR because of where motion picture media in general is going: VR/AR. For all the complaints we’re hearing about Ang Lee’s choice to do this, I think he’s done an important service to MoPic Science. By essentially giving us all a great experiment to learn from.

Jamie: I completely agree. I want more filmmakers to explore this medium. Has this technology been given a name yet, btw?

Juan (Juan Salvo, Colorist): Sorry, was literally training an AI to do my job

Pliny: Hey look everybody, Juan made it!

Does 120p help tell the story?

Pliny: Katie raises a good point  about MoPic Science, so I’ll use that as my segue into question #2: did Billy Lynn leverage HFR as a cinematic storytelling tool? Or is this just the world’s most expensive camera test?

Katie: I think that it was, in certain scenes.

Barbie: I think it’s still just “The Whole Shebang” or “HSTLA,” which stands for “Holy S*** That Looks Amazing”.

Jamie: I think the story of Billy Lynn was very mediocre, but I also think that the technology did not overshadow the story. What I did notice is that bad acting became unbearable, but good acting was really nice. Again, I think 120FPS is the TRUTH, which means every aspect of the filmmaking has to be on point.

Katie: Yeah, you could see so much that you could see when the actors were phoning it in.

Pliny: Wait, there were good performances?

Katie: I think that if we didn’t have “The Whole Shebang,” we would have forgiven the performances more.

Barbie: I agree with something Katie had expressed to me before…the war coverage was very compelling, and the dialogue scenes did not work at all. As Katie has pointed out, the HFR is used effectively as a tool here to convey a news footage feeling. It also brings all the gamers in the audience immediately. The first time when we flashback it’s actually to the soldiers walking in the market…I would argue that this transition is a gaming transition.

Jamie: I was bummed that there was very little war footage. I expected more, because I thought that was where the technology would shine.

Pliny: Jamie, I agree. Doug Trumbull gave an HFR presentation at AbelCine several years ago. He was only able to play back footage at 60p then, but it was a kung fu action scene, and it really did shine. The hypothesis was fast-paced action can get lots in a haze of motion blur at 24p, so that is where HFR is the most appropriate. Sure enought, the HFR doesn’t seem to “jump out at you” in fast-motion action sequences the way it does in dialogue scenes.

Katie: I also noticed though, it looked like there were takes they used because they had to, not because they were the best. That happens when you have 3D especially, there are takes that have [technical] problems, so you just can’t use them.

Jamie: Exactly. I think I paid more attention to that as well. I was thinking “why would they use this take?” I was picturing myself on set asking Ang “Are you sure you want to move on???”

Katie: Right. Having done plenty of 3D, I know that pain.

Juan: I can’t vouch for the acting, other than to say I’m not 💯 sure that lower frame rate would’ve really turned certain performances around.

Katie: No, but we might have forgiven a little. I would like to see the ability to switch frame rates scene to scene. Put the “cinematic”, or more traditional dialogue scenes at 24, then BANG 120 for the cool shit. That would be so amazing. And so impactful. And really be motivated by story.

Jamie: That could be interesting. But would definitely take the audience out of the movie.

Pliny: That does seem to be the end game—not just mixed frame rates within a single film, but mixed frame rates within a single frame. Think power windows, but for frame rate instead of color.

Jamie: Oooh now you are talking. Don’t forget HDR.

Pliny: I’m just repeating Mr. Trumbull’s ideas here.

Juan: A couple years ago we did some tests where we ramped up frame rates from 24 to 60. This was cool but also very distracting. I don’t know if it would work narratively. The frame ramping within the shot. You could also do it on cuts. I have also done mixed frame rates in one shot. This isn’t for faint of heart. There are all kinds of temporal issues. Only really works if you’re comping elements.

Jamie: re: temporal issues – I was thinking about how little the camera moved during the history of film, and only in the last 10-15 years have the camera’s really started moving erratically…this is different from steadicam and dolly shots – I mean EZ-RIG/handheld/Running…etc. The camera movements these days are all over the place and now we are really starting to pay attention to the stutter that appears from 24fps capture.

Juan: I wanted to touch on the broader point which is the question of the necessity of all these cutting-edge technologies in filmmaking and whether or not they’re in fact taking away from the audience experience diminishing the uniqueness of filmmaking. Biggest hits lately are all nostalgia rides. Do we even want to reimagine filmmaking with new tech?

Katie: We have to differentiate between a “good movie” and “good use of MoPic Sci.”

Juan: Shouldn’t new tech be new medium?

Jamie: Juan do you mean “Jaws” in 3D/HFR/HDR? What are we calling this tech?

Pliny: We’re calling this tech 120p-3D-4k-28-Ft-L-HDR. Catchy!

Does 120p Make Performances Worse, a.k.a. Is Everybody Actually A Terrible Actor?

Pliny: Well, the panel already stole my thunder for question #3, which is about the performances, and what it means to act for 120p. I actually did a little research on microexpressions, and they apparently last between 1/25 and 1/15 of a second. At 24p, that’s roughly one frame. So it could get lost in the noise. At 120p on the other hand, microexpressions are between 5 and 8 frames. So maybe when an actor is “Acting,” 120p is especially harsh. (Even harsher than Steve Martin’s nostrils.)

Jamie: Which is why the acting has to be on-point. You might need actors with broadway/stage experience. More-so than on screen actors.

Katie: But the intimacy was the problem for me, because like Pliny said—the “acting“.

Barbie: We may need to cast and screen test in 120?

Jamie: If the film intends to be 120, then casting should consider that. Some actors are good at “beats”, those actors would fail with 120.

Katie: There was also something uneasy for me with the closeups. They were too intense. So real, and so DAMN BIG in my FACE.

Pliny: That’s because the faces were in negative parallax. Notice that?

Katie: Yes! Noooooooooooo don’t doooooo that. It’s something we were doing (wrongly) when 3D was new and exciting, before we learned how to properly use our shiny new toys.

Juan: What’s more distracting in performances: 3D or HFR?

Jamie: HFR

Pliny: HFR

Barbie: HFR

Katie: HFR

Jamie: but that’s not fair. HFR with good acting is fine with me. HFR with mediocre acting is painful. HFR + 3D nostrils is fucking terrible. I think HFR won’t make acting better or worse, it will just show the truth. If your actor is giving you stellar performances, I think you might appreciate it more in HFR.

Barbie: I found most of the close-ups and shot-reverse-shot stuff extremely boring in BL. The group scenes in contrast were quite interesting. Pliny already knows my favorite scene is the Bravo company dining scene where the fracking business guy comes over and [the Sergeant] gives the best speech. This is the only truly subversive moment in the film. There are more faces to explore, more depth of field, i felt like my eyes were invited to move around the frame and “figure” out what is happening emotionally in the scene and how it’s shifting from moment to moment.

Katie: Barbie, one of the few scenes with good, natural acting.

Barbie: Last night at the BCPC panel with the BL post team, I was chatting with [Billy Lynn dailies technician] Derek Schweickart who pointed me to his MovieMaker Magazine article. In it he specifically says that in pre-pro they had considered switching between 24 for dialogue and 120 for rest, but they decided not to go that way because Ang Lee liked the intimacy of the faces in 120. I’m paraphrasing here of course.

Editor’s Note: BCPC held a separate panel event with the Billy Lynn post production team. The resulting podcast can be found here.

Juan: Billy is a bad movie, maybe it’s a good something else?

Katie: Juan and I keep butting heads over whether or not “filmmaking” should be traditional format/pure.

Juan: I think may be a purist. But I like new things! For the record.

Barbie: I dunno Juan. The more i sit around and think about it, the more i think i might end up a defender that BL is not in fact a bad movie.

Pliny: Barbie, I’m interested in your take on the directing. Because my reaction coming out of Billy Lynn was that it felt a bit like those Sundance indies where a first-time director hasn’t yet learned to get proper coverage.

That’s not a knock on Ang Lee—they gave themselves an impossibly aggressive 48-day schedule. Given the technical challenges, it’s reasonable to conclude that they just couldn’t get all of the coverage. (Like Jamie said earlier: “Are you sure you want to move on, Ang?”)

Did anyone else get that?

Jamie: Yep. It felt rushed. And I think Ang likes to come in on dialogue when it’s uncomfortable, which really doesn’t work for HFR.

Barbie: Yeah Pliny I know what you’re talking about re: indies. Stepping out of BL I couldn’t stop thinking about Lee’s early stuff, the right out of NYU. Eat Drink Man Woman, and films like that were extremely likable, but they had a messy, new quality to them…dare i say fresh.

Pliny: Wedding Banquet. So BL is a return to film school innocence?

Jamie: I don’t think it’s film school innocence. I think 3D/HFR is the truth, and if you don’t take the time on set to get everything right, you will regret it later.

Barbie: As a director i think he likes to see what happens, and that yields a wide variety of results. An experimenter. Sometimes you get a masterpiece (Lust, Caution), sometimes you get consummate managerial exercise (Life of Pi). BL was film school Lee. So agreed Pliny that it was rushed, underbaked, but maybe that’s not really a problem for me.

Jamie: You can’t hide from this format.

Juan: Sure you can! Hardly anyone saw the movie.

Katie: FYI, I’m not a film nerd. Never have been. I don’t have a “film” background. I’m a tech nerd, and I’m really in to exploring all that, in the bigger picture. So I have no attachment to Film, 24, or any of that. That’s why I love this film, despite its script/acting/etc. I love the exploration of tech.

I wish more people in our industry could see it, and basically nobody else.

Pliny: Well, taking a gander at the domestic box office right now, I think you might have gotten your wish, Katie.

Katie: 😂 fine by me! I really don’t want audiences to hate new tech. I want them to let us play and create and make mistakes until we have something amazing.

Barbie: I went to like a monday morning 10:30am show and there were waaaay more people than i expected actually. Like, 30 people even.

Pliny: So maybe BL is like that one Sex Pistols show in 1976 in Manchester, where only 30 people showed up but they all started really influential bands later on.

Jamie: Punk rock is a little different. Shooting HFR is 5 times the data, 10 times if you shoot 3D.

Pliny: 40 times if you shoot 4k!

Katie: I feel like Ang Lee “took one” for the greater good. It’s all part of an evolution. It’s not the end-point. If nobody does these things, we’ll never evolve.

What the hell is “Immersive,” anyway?

Pliny: Let’s talk a little bit about the “I-word”. All these new technologies — whether High Dynamic Range, Wide Color Gamut, stereo 3D or object-based audio — get slapped with the label “IMMERSIVE”. But then Rod Bogart had to go and say this on Twitter:

@iampliny @j_salvo @katiehinsen @refocusedmedia Everything in a movie is fake. "Immersion" increases your ability to see how fake it is.

— Rod Bogart (@RodBogart) November 22, 2016

Pliny: So is “Immersive” really immersive, or is Immersive the opposite of immersive? Discuss.

Katie: Rod is right—that was what was wrong with The Hobbit and why it was so hated-on. You have to pick the right story that people should be immersed in

Pliny: Wait, you’re saying there are stories you don’t want to be immersed in?

Katie: The right medium for the story.

Juan: I just want to add, the Hobbit was terrible on its own merits, not just the HFR. It was bad at 24fps.

Katie: Ok, ok.

Juan: But that’s just it, we’re talking about new tech being used in place of old tech. So we measure it by the expectations of the old tech. If you sold me on the BL “experience” I might have a very different opinion of it.

Pliny: Yes, new tech is always first used to recapitulate the old way of doing things. Only later on do we figure out how to use new tech to actually do new stuff.

Jamie: Should this new tech be compared to old tech though?

Barbie: I think there might be a threshold, where when there are enough elements of “immersion” it works. This threshold might be different for different people, but I would like it even better if BL was in VR.

Katie: There are good and bad uses of all stages of tech evolution. Some stories are right for certain techniques. Some aren’t. BL showed us that all in one film with its range of scenes.

Pliny: So we’re back to it being the world’s biggest camera test.

Jamie: I think it would be interesting to hear an average viewer’s thoughts on the movie and if they noticed any difference? Were average people watching BL?

Pliny: Felt like my screening was mostly civilians, yes. And some of them seemed genuinely moved.

Juan: Case in point: for very long time video games had terrible performances, but we still appreciated great games. Because we weren’t expecting fine acting.

Katie: What is immersion? We’re often talking about it with different definitions.

Pliny: For me, immersion is simply forgetting your surroundings and falling into the screen.

Barbie: I’m still trying to figure out why they made the decision to do the dialogue scenes with no depth of focus… against walls, against cars, etc. What percentage was budget/time, and what was a choice to limit distractions from the faces. As in, Billy and cheerleader are having their date backstage, so there is nothing in the background but curtain and wall. Boring, as opposed to scenes in the halftime show, plenty of depth of focus.

Katie: long DoF with stereo scan be problematic at worst, odd at best.

Jamie: There was one use of 3D in BL that took me out of the movie and it was a wide shot of the crowd and the very far BG became flat. It looked terrible.

Pliny: I’m a gamer. And I can’t say that I have ever felt HFR in games “jump out” at me they way it did when watching either The Hobbit or Billy Lynn. Even in dialogue-y cut scenes. Not sure why that is. FWIW.

Jamie: Pliny maybe that’s because you are immersed?

Katie: It goes back to immersion.

Pliny: Right. Games are immersive in a different way. The interactive element (you’re controlling the protagonist) creates a different dynamic. Cliche scenes—like when bad guys are chasing you across the roof of a moving train—are suddenly thrilling or terrifying again.

Katie: There are scenarios in which I don’t care to be “immersed”. There are scenarios in which I want to believe that this is a world in which I do not exist, I want to feel that this a fantasy world into which I have a window. e.g. Hobbit. I do want to be “immersed” in other scenarios: horror, wartime, etc

Jamie: Should the filmmaker decide that for their audience?

Katie: That’s what they’re there for. That’s storytelling.

Jamie: If we are going back to HFR being used selectively inside of a 24fps movie, then wouldn’t frame rate be akin to choosing a lens?

Barbie: How about this? What if all the dialogue scenes were done from a voyeristic POV? Is the trouble with the dialogue stuff in BL that they are covered in conventional 24fps film language?

Katie: ^yes!

Jamie: Good point.

Katie: I think you hit the nail on the head with exactly what’s “wrong” with BL.

Jamie: In that vein, how should they have shot BL then?

Katie: well… here’s the thing with “immersion”. In the stadium, I was, I suppose, walking behind them or with them. But in the house, I was a creepy asshole in the corner watching them in an intimate moment. I wouldn’t be in that scene, why would I? So it felt wrong to be “immersed.”

Barbie: I looooved the house scene. It was dreamy and disconnected…i think this was the one steadicam shot. It was so wrong it was right.

Katie: Barbie that’s why you’re a DP: voyeur.

Barbie: 100% accurate, Katie.

Jamie: The real test is: will porn pickup this tech?

Pliny: Only if the actors are really, really good.

Barbie: This will sound glib, but the people out there who watched the 3D porn, they will like BL.

Pliny: That may just be the boldest statement of the evening.

Final Thoughts.

Pliny: Time to wrap up. One final thought, everyone?

Jamie: This format needs a fucking name. A short name.

Katie: Don’t stop experimenting with new tech. Don’t shit on it when people do it. And seriously, screw celluloid, it’s a pain in the ass. We’ve solved so many problems with digital technology, and even more with HDR and HFR.

Barbie: I think Jamie nails the final thought.

Juan: 1) the first cars were absolutely terrible means of transportation 2) they were mistakenly called “horseless-carriages”. In hindsight, issue 1 was a product of ambition getting ahead of technology; issue 2 was a product of imagination falling behind ambition.

Pliny: My final thought: those scrolling end credits looked damned smooth. 120p is just cheating.

Katie: YASSSS I thought of you

Barbie: 💯

Pliny: Thanks everybody. This has been great.

John “Pliny” Eremic created Endcrawl.com, edits this blog, and runs workflow at HBO. @iampliny

Katie Hinsen is an award-winning finishing artist and founder of the Blue Collar Post Collective. @katiehinsen

Barbie Leung is a New York-based cinematographer. (barbieleung.com)

Juan Salvo is principal at theColourSpace. @j_salvo

Jamie Metzger is a DIT with credits including Furious 7 and Transformers. JamieMetzger_600