👡 🐠 🤜🏼 Sol Levante: how they did anime in 4K and HDR for Netflix 🤚🏻 👩🏼‍✈️ 📬

Anime, often rejected as just cartoons, is a variety of fairy tales, lively style and elegant graphics. This is a form of art that over the past fifty years has developed and grown in popularity, diversity and complexity. It probably began with painted lamps at the beginning of the 20th century, then in the 1970s it gained wide popularity in Japan. Today, viewers have a choice of hundreds of TV shows and feature films in anime style from around the world.

Our Creative Technologies team wanted to improve the technical quality of the anime picture, find out what new creative possibilities this can give people, and what it will take to increase the resolution of anime from HD to 4K and enter a wider color palette in HDR into the artists ’toolkit(high dynamic range). When it came to 4K, most Japanese animators asked for the same thing: larger paper! However, the artists at Production IG believe that the future lies in digital technology. This is how the large and colorful collaboration was born - and the resulting Sol Levante short film is available for viewing on Netflix in 4K Dolby Vision and Atmos!

Netflix's Creative Technologies team is trying to figure out how to improve future-proofed content creation and is working with various research and development partners to make these improvements available to artists in real workflows. Our central tasks are both combining high quality and durability with reliability of presentation, and creating a space for creativity. Given the rapid proliferation of consumer devices with 4K HDR display capabilities, it’s easy to imagine that in five years this quality will become the standard.

Over the past few years, we have successfully remastered such works as Knights of Sidonia, Flavors of Youth and Godzilla, raising the quality from SDR to HDR. But what if you immediately increase the working resolution and create an anime initially with an eye on HDR? How will this affect creative decisions? What problems will arise, of a creative and technical nature? How will the budget and terms change? What new creative possibilities can this give?

Tokyo-based Production IG, which has established itself as Ghost In the Shell level projects and animation in Kill Bill, has assembled an experienced team that has mastered the work completely “in digital” and also wanted to answer these questions. Our common curiosity led to the creation of an experimental short-length anime in 4K HDR Immersive Audio called Sol Levante. And to help the industry deal with the 4K HDR and immersive audio technologies in the anime, we released all materials for free access , for download and experimentation.

Director Akira Saito works at After Effects

Akira and Haruka work together

Current Anime Status: Business

Although the incredible stories of modern anime never get old, the workflow itself, born in Japan, has not changed for many years. To understand the need for our experiment, it is important to understand how anime is actually created in Japan.

Since the beginning of the century, the anime industry has moved away from the traditional system, when the anime studio owns most of the intellectual property, and has come to the modern business model of the “production committee”. Although it has become easier for studios to create content thanks to financial support, increasing the number of participants complicates the process.

Partially or completely, the production is subcontracted to several companies or freelancers, and this can happen at several levels. With such a large number of subcontractors, it is quite difficult to introduce everyone to changes in technology and new creative opportunities. In different projects, the number of freelancers is different, however, most of them work at home, and their equipment is inferior in quality to what the studio can afford.

Current Anime Status: Production

In addition to the complexities of the studio system, additional difficulties arise when trying to innovate the workflow of creating anime. And while 3D CG is gaining in popularity - shows such as Saint Seiya and Ultraman are entirely computer generated by 3D artists - most of the anime is still hand-drawn. Some artists prefer the feeling of paper, or they don’t have the time and money to invest in equipment that allows them to try their hand at digital art. The workflow after the drawing is already digital, however, to support a larger resolution, the equipment needs to be seriously updated. There is also a shortage of teachers with experience and tools that would enable them to educate a new generation of digital animators. Hence the lack of staffable and willing to work with a number already at the stage of drawing.

To go on air, anime is usually done in the resolution of 1280x720, or “Half HD”. Only the most significant shows are shown in 1080 HD. To switch to 4K, switching to a number becomes necessary, because the pictures need to be scanned. But if a regular-sized paper sheet is simply scanned by increasing DPI, then unnecessary details will come out in pencil strokes. Therefore, hand-drawn anime requires larger paper.

Also in the process of anime rendering, color management is rarely used - a process that helps to obtain a predictable and consistent color of drawings at each step of production and post-production. With a few exceptions, everything is created in the sRGB color space - the most popular spectrum used in computer graphics. In feature films, animators usually paint in sRGB, and in post-production they add 3D LUT to turn it into P3, which preserves the appearance of sRGB in the P3 color space.

You may also be surprised at the fact that Japanese broadcasters set the color temperature to 9300 K, much bluer than the temperature of 6500 K, which dominates the rest of the world. Many of the animes licensed for Netflix converted to D65 BT.1886 at the very end of post-production to meet our specifications and look right on our streaming service - as intended by their creators.

Understanding all the difficulties involved in transferring anime from paper sheets to a fully digital workflow, Creative Technologies worked closely with Production IG to try to predict some of the problems. However, only when the team began to really use digital styluses and tablets, did these discoveries begin to take on a clear form.

Sol Levante: discoveries during post-production

In the current process of making anime, it usually takes from a year and a half to two years from writing the script to completing work on the 12-episode season of the series. In the case of our three-minute Sol Levante short, director Akira Saito was engaged in a storyboard for two weeks. She tried to incorporate many different lighting options into the film, varying from scene to scene. The story begins at dawn, develops during a gloomy day, then continues at night and ends with sunrise. Akira said that she was very interested in working, and that she was enthusiastically engaged in animation, not suspecting what problems would appear in her production.

One example of a storyboard

When Akira thought about switching to 4K and increasing the resolution, she achieved a good balance between frames with simple drawings and frames with many details. In many frames, she created a world whose density is high enough to impress the audience, and at the same time it has small details that can be seen after enlarging the picture.

During the character development phase, the animator Hisashi Ezura spent a lot of time on fine-tuning the thickness of the lines due to the expansion of the dynamic range. Outlines of characters - a phenomenon unique to anime, but if you use the usual thickness of the lines, they will look too sharp in the SDR and too unnatural in the HDR. He set the stylus to a “softer” tap on the canvas to change the appearance of the lines.

Test for selecting the thickness of the contours

In the color development phase, the team initially decided that a brightness of 300 nits would be enough for pre-production. However, seeing the impression that 1000 nits make on the Eizo CG3145 monitor, they felt that the gap was too large, and 300 nits would not allow the designer to use the entire palette. Since in the anime color decisions are made at the pre-production stage, it was very important for Miho Tanaka, the color designer for Production IG, to see how the brightness of 1000 nits behaves on the mastering monitor.

It was also quite difficult to choose a skin tone. A special and thorough adjustment requires the color of the shadow, since it can greatly affect the overall perception of the face and the character as a whole. Miho spent a lot of time making the characters look beautiful at any time of the day. HDR made it easier to choose colors for dark scenes, because in this mode the picture does not start to look “dirty”, and the lines remain clear. Without a decent dynamic range, it is difficult to maintain the clarity of such thin lines, such as eyelashes. HDR allows you to clearly separate the colors from each other, so that they do not mix in an undesirable way.

During the study of the appearance, other problems appeared, as well as opportunities. Thanks to HDR, the typical visual effect of eye gloss can be replaced by another method of using colors. If you add bright lines to the lips, they look shiny. As a result, the director processed all the feedback and accordingly adjusted her design and colors.

On the left - the test version, on the right - the final

One of the most difficult problems with HDR during pre-production was the limitations associated with design and drawing tools. The interface of the graphic programs was too bright to constantly look at it at a brightness of 1000 nits, and the background had to be changed to gray (180) from white (255) during preview on an HDR monitor. Many graphics tools still do not support the full 16-bit output critical for HDR. And the color picker looks different on SDR monitors and HDR monitors, which makes it difficult to select colors accurately.

Also at this stage, problems were found with a resolution of 4K. When drawing lines, artists constantly went through the process of enlarging the drawing and checking the result of their work due to restrictions on the resolution of their screen or tablet. When viewing the whole picture, the lines became too thin to consider all the details of the image.

Sol Levante: discoveries in the production phase

At the pre-production stage in our collaboration with Production IG, the real difficulties began to appear in the production phase.

To begin with, the team had to conduct many experiments in order to properly configure the information transfer system and select the appropriate color profiles, as well as correctly show the colors on each monitor, moreover, depending on the specific program. Each of the team of artists needed several monitors in order to look at the picture in SDR and HDR modes, and to continue supporting the projects that they were already working on in SDR mode. The problem of color management in animation affects many stages of production. This project was especially complex, but Junichiro Aki and Katsushi Eda, together with color management specialist Masakazu Morinaka, created a working system.

For Sol Levante, absolutely everything was done in digital. In addition to using the Procreate app for iPad in the pre-production and generation of ideas, Production IG also used ClipStudio to build intermediate images, Vue to draw backgrounds and selected elements, Retas Stylus to work with color, Photoshop and After Effects. Animators experimented with Toon Boom Harmony, which supports the extremely popular "animation of clippings" technology outside of Japan.

Sol Levante Workflow Diagram

The team immediately realized that some of the anime techniques that they had relied on for years would not work with increasing brightness and expanding the spectrum. For example, a typical effect in the anime is a smooth transition to 100% white, which is used as one of the methods of narration. However, the white in the HDR turned out to be too bright, so the artists decided instead to overlay a white layer on top. And every time something did not work as expected, they found a new way to achieve the same result.

4K resolution had the greatest impact on the workflow when generating the final frames with compositions consisting of images much larger than 4K in size - they were necessary for panning. And this turned out to be such a serious obstacle that the entire project was slowed down for several months. Some of the problems that caused it appeared due to the fact that the equipment needed to be configured to work with 4K; others lay in how the software works and how it uses system resources. For software manufacturers, the main problem remains the need to improve work with resources.

When creating patterns in numbers, you can save thin lines, because you do not need to scan paper to digitize. Compared to drawing on paper, the animator was no longer so worried about the size of the canvas. He also created a “settings table” that limits the level of detail of characters and their accessories under certain conditions. The amount of detail depends on the size of the character on the screen. Here you need to decide which details to add for resolution in 4K, and which should not be wasted because they will not be visible.

Production IG settings table

Continuing to work on Sol Levante, the Production IG team decided to outsource part of the intermediate rendering (creating intermediate frames between the two main images). Outsourcing similar work to other companies is a common practice in the production of anime. This was quite difficult to do, since subcontractors usually reject such new ideas as digital workflow - in order to adapt to these new tools, investments in equipment and retraining of personnel are needed. In addition, Production IG decided that for the right choice of colors, drawing lines and building compositions, it was absolutely necessary to constantly work on an HDR monitor with a brightness of 1000 nits. But in the post-production market there is a constant shortage of HDR monitors, especially at affordable prices.which from the very beginning greatly affects the decision to work with such monitors.

As a result, Production IG nevertheless managed to find a sufficient number of subcontractors with a fully digital technological process, with one exception: the breakdown of frames by time, or timeshit, which is used to arrange drawings on a timeline and mark places for intermediate rendering, camera movements and other technical information. Timeshield was the only paper document in the entire Sol Levante manufacturing process.

Usually, anime does not use color grading at the last stages of production, but while working on Sol Levante, animators worked with an image container (PQ) with a brightness of 10,000 nits, and it was displayed on a monitor with 1000 nits, so it was necessary to make a general pass to align the visible spectrum of colors and general appearance of the picture. During the calibration, the colorist with the director discovered several possibilities to improve the final picture - for this it was necessary to change some of the initially selected colors, and allow certain parts of the image to stand out against the background. For example, a colorist tweaked the color of lightning in one of the frames so that it stands out more and added granularity during the eruption of the volcano to highlight the texture of the image.

Since the entire PQ spectrum was used in production, the colorist had more creative freedom and the ability to adjust colors during graduation, and in addition, the finished project will be easier to remastere in the future. It is usually customary to rely on the original color, but this experience has shown the Production IG team that more creative decisions can be made in the digital technical process.

Of the entire production, one of the most valuable lessons for the director was that the studio's managing process should really manage everything that happens, and instead of an infinite variety of possibilities, force subcontractors to use a specific technical process and certain tools. However, little can change until large studios unite with equipment manufacturers and make a large-scale transition to digital.

Anime Sound: Immersive Sound

The sound landscape of anime is different from all other types of media. Music and sound are critical aspects of storytelling, and Sol Levante was an ideal opportunity to demonstrate the immersive sound and experimental picture working together in 4K and HDR. Our team believes that immersive sound is the natural next step in the evolution of audio, as it enhances both the quality and the creative possibilities of storytelling. The combination of 4K, HDR and immersive sound allowed to breathe life into the world of Akira. And all this is possible with the same tools that mixers are already used to using.

Mixer Will Files and sound designer Matt Yokum, together with Akira, voiced her world in the Dolby Atmos standard. Haruka was involved in this process at an early stage, when drawing was still underway, and helped transfer Akira's ideas from Japanese to English, while preserving the creative nuances and meaning.

In one case, Matt added a crow croaking in a scene where hundreds of birds fly up, and as a result, Akira asked him to find the sound of a bird that would be found in Japan. Matt created a completely new sound based on the cry of a local bird, and in this way an effect was obtained that would not have existed without universal collaboration.

Using Atmos inside Pro Tools, Will achieved a contrasting sound picture and managed to weave into it an orchestral recording of music by composer Emily Rice, recorded on a Schoeps ORTF 3D microphone during dubbing. While watching a video, sound moves around the room, flies up to the ceiling, drops to the floor, and takes off again, and thus tells the dramatic story of Sol Levante without dialogue.

Mixing immersive sound is a new concept for many mixers, including Japan, where there are still not many cinemas that can play such sound. However, mixing in Dolby Atmos allows the creative team to create a single mix, which can then be used to get all the other options, from 7.1 and 5.1 to stereo. This format is ideal for the archive, and is able to bring a new dimension to the anime.

Sound Supervisor and Recording Mixer Wil Files Works with Dolby Atmos

The future of anime production: what's next?

The delightful world of Sol Levante is the culmination of art, technology and curiosity. Considering everything that we learned during these two years of working with Production IG, we want to offer animators and creative teams technology development, work together with equipment manufacturers to better support the anime industry, and work with anime studios to apply our experience to their products. To help the industry better understand 4K HDR and immersive sound in the anime, we uploaded all the working materials used in Sol Levante so you can download and experiment with them. Subscribers today can watch Sol Levante on Netflix. Best movie will be perceived on a device configured for HDR with a premium subscription.

From left to right: Katsushi Ed, Miho Tanaka, Akira Saito, Hisashi Ezura, Masakazu Morinaka.

Director Akira Saito told us that “4K HDR allegedly gave us wings and a motor so that we can see new horizons and the advent of a new era. “We continue to set ourselves challenging goals and innovate for the future.” In this project, we encountered many difficulties, but Akira believes that for the 4K HDR studio is the only way to stay at the forefront of content creation.

However, the current situation is such that technical difficulties in the transition from HD to 4K do not disappear. We encourage creative people to go beyond the boundaries of permission by telling their story. We want to create content in a format in which our viewers will consume it, and we want it to maintain an excellent appearance for as long as possible. However, we cannot ignore the fact that in the case of anime, tools and equipment are still not designed to easily organize the technical process of creating images in 4K. Much remains to be done and studied, and the Creative Technologies team is working closely with software and hardware manufacturers to share their knowledge and try to accelerate improvements in the industry.

Adjustment for HDR is much simpler, because here the problems are less complicated. There are already software manufacturers making changes to their programs due to the problems we encountered in Sol Levante production - such as different display of the color picker on different monitors, and a dazzling interface. The biggest hurdle for the entire industry remains the lack of affordable HDR monitors. In the end, even with a properly working tool, to select a color in HDR you will need a monitor on which you can see what color you choose. As soon as there are more of them, then the widespread use of HDR in anime will become only a matter of time.

“Who would not want to live in a bigger house?” - so the animator Hisashi Ezura told us about working with HDR, and this analogy made Haruka laugh, since everyone knows in which small apartments the Japanese are hiding, constantly dreaming of a larger house. But this dream remains a dream. And his dream has come true. And now that the HDR has given him access to all these new features, he will never come back.

In Italian, Sol Levante means “the sun rising in the east,” which metaphorically means Japan. Akira chose this name because for her the whole project was dedicated to the beginning - the beginning of something new for Japanese animators. And the last frame of the film is dawn, because the film itself is the dawn of a new technological era for anime!