Vertical Divider
Consumers Left Behind by the Film Industry in Adapting HDR
Virtually every TV sold in 2021 promotes some form of HDR, even if the spec says HDR ready. Basically, the chips and the software are available, so it costs very little to add the capability. But ask a random set of 10 people you know, to define it and you’re unlikely to get a reasonably accurate response. Ask them if they’ve seen the effects of HDR and the responses are worse, yet. This article shows how much effort the film industry’s TV segment is making to improve the content. Most consumers are reluctant to change the basic settings on their TVs. Asking them to make selections from a menu of three HDR standards is absurd. To take advantage of the benefits of HDR, set makers must standardize and make it easy for the consumer to select the proper settings, otherwise all the work by the artists and colorists will be for naught
Rory Gordon, senior colorist at Santa Monica-based post-production facility Arsenal FX Color, wrote a paper in 2019 for the SMPTE Motion Imaging Journal on the subject of how high dynamic range (HDR) can impact the creative aspects of storytelling, by emphasizing that “workflow has a strong impact on the creative experience,”. That remains true, Gordon insists, with the only thing changing since 2019 is that “people are now more aware of HDR and have a deeper understanding of what an HDR image can look like today.” As a senior colorist who works primarily on television and streaming content of all genres, Gordon emphasizes that it is “really important” to comprehend the engineering process, and the technical side of the creative work that people like her do every day. If a colorist is creative and can understand the way color scientists look at the big picture, they will be more productive. It might be a problem with out-of-gamut mapping, for example, that affects extreme edge points like highlights—ambulance lights in an image, for instance. They need to understand the tradeoffs that the color scientists made in designing that color management system to preserve parts of the image that are generally more important, like skin tone.” Gordon adds that it is important to understand where imagery is being viewed in a wide range of environments and on a seemingly endless wave of devices and monitors, those factors have the potential to change an image, or at least, how it emotionally impacts the viewer. So, the subtle aspects of color grading and technical tasks like managing metadata become more crucial than ever.
“The reason HDR is so impactful is not necessarily because of the imagery itself,” she says. “It’s because having brighter, more colorful pictures enable viewers to watch the images in a greater variety of surrounds, while still having a quality experience. There has been a lot of work done on how a surround will change an image. So, the idea that a greater contrast ratio is extremely impactful is not because more contrast will produce a pleasing image. It’s that the image has a greater chance of being perceived as contrasty in brighter surroundings.
“… color gamut can be a bit of a different ballgame because so few home displays today fully Rec. 2020. But most displays cover the P3 color range now, similar to a theatrical experience. Bringing in a more expansive image that feels theatrical because of the greater color gamut can make the content more special in the home.”
Contrast or color relationships in content should hold up on many different kinds of displays and in many different kinds of environments as the challenge of “relational color.” The rapid advancement toward greater HDR capabilities in consumer display technology has created new challenges for both the engineering and creative communities to consider in designing and executing post-production workflows.
“People are working on these issues,” she points out. “You have ambient light calculations and implementations being put to use on displays. With Filmmaker Mode on LG OLED some formats i.e., Dolby Vision, with the metadata doing a handshake with the display in switching over to the settings. With Filmmaker Mode—that one just straight-up grays out some settings. The general idea is to make things easier for consumers. They may or may not like it, and that’s their choice, but it does make it easier for the intention that was set by filmmakers in a high-end color bay to be faithfully represented at home.”
Both artists and consumers will benefit from learning about the huge range of home-viewing options. “These displays can now go up to 1,000 nits and beyond and to P3 and beyond,” she says. “So, the piece that is really necessary for having artist produce the content and viewers enjoy content is to have them understand what these different modes do. Metadata is changing for different platforms and there are cool new tools that can make particular tweaks to make it easier to marry a standard def version with an HDR version.
I personally think that if you can find ways to give people a little bit of new information from time to time, they can use it in a way that shows they are smarter than we give them credit for. To a degree, we have to be more generous with the benefit of the doubt in assuming viewers will be able to make good decisions with their home displays, and the same goes for artists and color scientists. After all, you don’t gain mass implementation of a new technology just by having the technology be good. You have to convince people that there is a way and a good reason to learn how to use it.” For artists like Gordon, achieving the creative intention set by filmmakers so that it can be impactful on viewers no matter what viewing platform or format they are utilizing largely revolves around figuring out the best way to be “color target independent,” a goal that still remains, at least for now, somewhat elusive. By “color target independent,” Gordon means that the color workflow should ideally be able to utilize highly complex metadata in a seamless way that lets, for example, HDR mastered content be viewed in standard definition, or other formats, when necessary.
Gordon emphasizes that HDR and SDR “are married now” through metadata and through the basic color workflow process. “That’s the main thing that has changed in production,” she says. “We are now looking at how a workflow be color target independent? When we work on shows right now, we work in a log space, or in the ACES world, what we would call ‘scene referred space.’ The intention there is to be able to make creative decisions that are relevant and continue to be applicable regardless of whatever the color target is. That can be difficult when you consider that in the SDR world, if an image clips, it’s not a huge deal if you lose some detail because the dynamic range isn’t sufficient to make it distract. But that same shot, in the HDR world, can be distracting. “In the HDR version, when you get bright highlights that are 1,000 nits instead of 100 nits, you do see those details that are frequently distracting. Things like gels or filaments in a light or wrinkles in a cyclorama, for example. But, because we have the knowledge that those flaws will be visible at some point, we are typically able to finish even in SDR correctly so that they won’t be, because we can still recognize errors on our vector scopes and waveform monitors. So, the idea is that in our workflow, we need to have a way to translate the relationships, like contrast ratio—we want to make sure our pipeline is portraying it in roughly equivalent relationship between SDR and HDR.” Currently, there are three primary HDR flavors offered with modern televisions, and most HDR-capable content is mastered these days using one of them. Gordon says most of her television work is finished in Dolby Vision—a format widely supported by most high-end 4K and OLED televisions. Dolby Vision content travels with dynamic metadata to a user’s television, which lets the viewing device adjust color and peak brightness levels frame-by-frame. It is capable of producing up to 10,000 nits of peak cinema-style brightness, though few televisions are yet fully capable of taking full advantage of that feature.
All three improve viewing brightness, shadows, and depth on modern televisions capable of receiving their input, and likely further HDR-related improvements for consumer viewing will come down the road. Gordon points out that the industry is still far away from any type of official standard, and she doesn’t expect there to be one any time soon. “Certainly, it would probably put less pressure and stress on productions and finishing houses if there was one single common HDR format, but I think this situation differs from the shift from SD to HD or from HD to 4K, because essentially there is only one core item—the image itself. With HDR, you have the image and the dynamic metadata that has to travel with it. There is no real pressure coming from the consumer to standardize, because the viewer can get really great images right now whether it comes in the form of Dolby Vision, HDR 10, or HDR 10+.
Gordon doesn’t see the fundamentals of her creative job radically changing as HDR proliferates and evolves. She thinks that camera tests with wardrobe, makeup, and production design are becoming more crucial than ever, and certainly there are “more gotchas” along the way—“more holes to fill, like times when you have to generate an SDR file first and times when you have to generate an HDR file first, which can impact your production schedule. And there are certain technical hurdles with certain types of images, like trimming lightning, for example. There, if you are working in Dolby Vision, it might require more attention, because with dynamic metadata, the system can take the peak brightness and shadow on average per shot, but if the scene is supposed to have a bunch of flashing lights, you have to find ways to instruct the software to allow certain frames to be brighter, darker, brighter, and so on. So, there is some juggling with more balls than in the past, but overall, it is still juggling. But as has always been the case, composition of the image is the most impactful thing on color. It still goes back to cinematography, production design, locations, wardrobe—things like that.” As the industry trends further toward HDR and related developments, Gordon says she has self-educated herself on these trends, including consuming a series of books she highly recommends on this topic. Among them are “Color Appearance Models” by Mark D. Fairchild; “The Reproduction of Color” by R.W.G. Hunt; “The Art and Science of HDR Imaging” by John J. McCann and Alessandro Rizzi.
Virtually every TV sold in 2021 promotes some form of HDR, even if the spec says HDR ready. Basically, the chips and the software are available, so it costs very little to add the capability. But ask a random set of 10 people you know, to define it and you’re unlikely to get a reasonably accurate response. Ask them if they’ve seen the effects of HDR and the responses are worse, yet. This article shows how much effort the film industry’s TV segment is making to improve the content. Most consumers are reluctant to change the basic settings on their TVs. Asking them to make selections from a menu of three HDR standards is absurd. To take advantage of the benefits of HDR, set makers must standardize and make it easy for the consumer to select the proper settings, otherwise all the work by the artists and colorists will be for naught
Rory Gordon, senior colorist at Santa Monica-based post-production facility Arsenal FX Color, wrote a paper in 2019 for the SMPTE Motion Imaging Journal on the subject of how high dynamic range (HDR) can impact the creative aspects of storytelling, by emphasizing that “workflow has a strong impact on the creative experience,”. That remains true, Gordon insists, with the only thing changing since 2019 is that “people are now more aware of HDR and have a deeper understanding of what an HDR image can look like today.” As a senior colorist who works primarily on television and streaming content of all genres, Gordon emphasizes that it is “really important” to comprehend the engineering process, and the technical side of the creative work that people like her do every day. If a colorist is creative and can understand the way color scientists look at the big picture, they will be more productive. It might be a problem with out-of-gamut mapping, for example, that affects extreme edge points like highlights—ambulance lights in an image, for instance. They need to understand the tradeoffs that the color scientists made in designing that color management system to preserve parts of the image that are generally more important, like skin tone.” Gordon adds that it is important to understand where imagery is being viewed in a wide range of environments and on a seemingly endless wave of devices and monitors, those factors have the potential to change an image, or at least, how it emotionally impacts the viewer. So, the subtle aspects of color grading and technical tasks like managing metadata become more crucial than ever.
“The reason HDR is so impactful is not necessarily because of the imagery itself,” she says. “It’s because having brighter, more colorful pictures enable viewers to watch the images in a greater variety of surrounds, while still having a quality experience. There has been a lot of work done on how a surround will change an image. So, the idea that a greater contrast ratio is extremely impactful is not because more contrast will produce a pleasing image. It’s that the image has a greater chance of being perceived as contrasty in brighter surroundings.
“… color gamut can be a bit of a different ballgame because so few home displays today fully Rec. 2020. But most displays cover the P3 color range now, similar to a theatrical experience. Bringing in a more expansive image that feels theatrical because of the greater color gamut can make the content more special in the home.”
Contrast or color relationships in content should hold up on many different kinds of displays and in many different kinds of environments as the challenge of “relational color.” The rapid advancement toward greater HDR capabilities in consumer display technology has created new challenges for both the engineering and creative communities to consider in designing and executing post-production workflows.
“People are working on these issues,” she points out. “You have ambient light calculations and implementations being put to use on displays. With Filmmaker Mode on LG OLED some formats i.e., Dolby Vision, with the metadata doing a handshake with the display in switching over to the settings. With Filmmaker Mode—that one just straight-up grays out some settings. The general idea is to make things easier for consumers. They may or may not like it, and that’s their choice, but it does make it easier for the intention that was set by filmmakers in a high-end color bay to be faithfully represented at home.”
Both artists and consumers will benefit from learning about the huge range of home-viewing options. “These displays can now go up to 1,000 nits and beyond and to P3 and beyond,” she says. “So, the piece that is really necessary for having artist produce the content and viewers enjoy content is to have them understand what these different modes do. Metadata is changing for different platforms and there are cool new tools that can make particular tweaks to make it easier to marry a standard def version with an HDR version.
I personally think that if you can find ways to give people a little bit of new information from time to time, they can use it in a way that shows they are smarter than we give them credit for. To a degree, we have to be more generous with the benefit of the doubt in assuming viewers will be able to make good decisions with their home displays, and the same goes for artists and color scientists. After all, you don’t gain mass implementation of a new technology just by having the technology be good. You have to convince people that there is a way and a good reason to learn how to use it.” For artists like Gordon, achieving the creative intention set by filmmakers so that it can be impactful on viewers no matter what viewing platform or format they are utilizing largely revolves around figuring out the best way to be “color target independent,” a goal that still remains, at least for now, somewhat elusive. By “color target independent,” Gordon means that the color workflow should ideally be able to utilize highly complex metadata in a seamless way that lets, for example, HDR mastered content be viewed in standard definition, or other formats, when necessary.
Gordon emphasizes that HDR and SDR “are married now” through metadata and through the basic color workflow process. “That’s the main thing that has changed in production,” she says. “We are now looking at how a workflow be color target independent? When we work on shows right now, we work in a log space, or in the ACES world, what we would call ‘scene referred space.’ The intention there is to be able to make creative decisions that are relevant and continue to be applicable regardless of whatever the color target is. That can be difficult when you consider that in the SDR world, if an image clips, it’s not a huge deal if you lose some detail because the dynamic range isn’t sufficient to make it distract. But that same shot, in the HDR world, can be distracting. “In the HDR version, when you get bright highlights that are 1,000 nits instead of 100 nits, you do see those details that are frequently distracting. Things like gels or filaments in a light or wrinkles in a cyclorama, for example. But, because we have the knowledge that those flaws will be visible at some point, we are typically able to finish even in SDR correctly so that they won’t be, because we can still recognize errors on our vector scopes and waveform monitors. So, the idea is that in our workflow, we need to have a way to translate the relationships, like contrast ratio—we want to make sure our pipeline is portraying it in roughly equivalent relationship between SDR and HDR.” Currently, there are three primary HDR flavors offered with modern televisions, and most HDR-capable content is mastered these days using one of them. Gordon says most of her television work is finished in Dolby Vision—a format widely supported by most high-end 4K and OLED televisions. Dolby Vision content travels with dynamic metadata to a user’s television, which lets the viewing device adjust color and peak brightness levels frame-by-frame. It is capable of producing up to 10,000 nits of peak cinema-style brightness, though few televisions are yet fully capable of taking full advantage of that feature.
- HDR 10, the mandatory format for Blu-ray discs, sends static metadata into a video stream to calibrate the overall picture, but not on a frame-by-frame basis, and only goes up to 1,000 nits of brightness.
- HDR 10+ also relies on dynamic metadata and can go up to 4,000 nits of brightness.
- Both Dolby Vision and HDR 10+ support 12-bit color depth.
All three improve viewing brightness, shadows, and depth on modern televisions capable of receiving their input, and likely further HDR-related improvements for consumer viewing will come down the road. Gordon points out that the industry is still far away from any type of official standard, and she doesn’t expect there to be one any time soon. “Certainly, it would probably put less pressure and stress on productions and finishing houses if there was one single common HDR format, but I think this situation differs from the shift from SD to HD or from HD to 4K, because essentially there is only one core item—the image itself. With HDR, you have the image and the dynamic metadata that has to travel with it. There is no real pressure coming from the consumer to standardize, because the viewer can get really great images right now whether it comes in the form of Dolby Vision, HDR 10, or HDR 10+.
Gordon doesn’t see the fundamentals of her creative job radically changing as HDR proliferates and evolves. She thinks that camera tests with wardrobe, makeup, and production design are becoming more crucial than ever, and certainly there are “more gotchas” along the way—“more holes to fill, like times when you have to generate an SDR file first and times when you have to generate an HDR file first, which can impact your production schedule. And there are certain technical hurdles with certain types of images, like trimming lightning, for example. There, if you are working in Dolby Vision, it might require more attention, because with dynamic metadata, the system can take the peak brightness and shadow on average per shot, but if the scene is supposed to have a bunch of flashing lights, you have to find ways to instruct the software to allow certain frames to be brighter, darker, brighter, and so on. So, there is some juggling with more balls than in the past, but overall, it is still juggling. But as has always been the case, composition of the image is the most impactful thing on color. It still goes back to cinematography, production design, locations, wardrobe—things like that.” As the industry trends further toward HDR and related developments, Gordon says she has self-educated herself on these trends, including consuming a series of books she highly recommends on this topic. Among them are “Color Appearance Models” by Mark D. Fairchild; “The Reproduction of Color” by R.W.G. Hunt; “The Art and Science of HDR Imaging” by John J. McCann and Alessandro Rizzi.
Contact Us
|
Barry Young
|