Photorealistic augmented reality graphics in TV

Augmented reality graphics offer myriad opportunities for the broadcast industry in terms of viewer experience, storytelling and data visualization. Ahmet Oturgan, Business Development Manager – South Europe at Zero Density, explains how they help to tell stories with stunning visuals and offer interactive ways for the audience to be included in the story.

Augmented reality graphics can be used to educate or show people about anything from significant historical events to dangerous weather conditions et election data. You can enrich your content with AR by depicting what the situation or impact was, is or will be, so audiences get switched on to the content.

AR is remote working friendly. Production teams do not need to travel to different locations to get related footage. You can sit in your production studio and use AR to show audiences anything you desire. You can even “teleport” actors and presenters from across the world to appear on a graphic overlay.

Changing physical sets might prove difficult but AR provides flexibility to give your existing studio a makeover as you can even edit the graphics at the last minute and have changes instantly appear in your final scene. This also reduces time spent on set.

Gaming solutions like Moteur irréel are powerful enough to render huge landscapes in real time. Now more than ever, we see broadcasters around the globe employ AR graphics for daily shows and eye-catching segments. Augmented reality (AR) in TV broadcasts is nothing new; the level of photorealism is.

Photorealism and Storytelling

Broadcasters cover a wide range of colossal topics such as breaking news, economy, elections, sports and weather. Demonstrating these topics in an understandable and pleasing-to-the-eye way is the ultimate alliance that will ensure the audience is intrigued and engaged. Augmented reality enters the scene as a fast, accessible, and effective technology. Moreover, when the storytelling is photorealistic the quality of the broadcast reaches new heights.

Recently, at IBC Show (as in NAB 2022), Zero Density gave a detailed demonstration of how photorealistic augmented reality graphics can help diffuseurs enhance everything from viewer experience to narration et data-driven workflows.

Creating virtual graphics that are realistic to the human eye requires live rendering capacity, realistic live shadows, and realistic live reflections.

Rendering

We have come a long way for broadcast renderers. The Zero Density team became the first to adopt a game engine for broadcast purposes back in 2016. The company transformed Unreal Engine by modifying its source code to fulfill the requirements of real-time production and developed real-time node-based compositor and real-time image-based keyer. It provides all necessary tools to the users to be able to create stunning visuals at highest fidelity and stability with unlimited flexibility.

When the results were demoed at IBC 2022, people were surprised by the photorealistic results and asked where the racks of equipment needed to create the composite were kept. It was a big surprise for them to see all the processing including video I/O, keying, rendering and compositing in a single workstation. Today, many noticed that game engines are rendering more photorealistic graphics than broadcast engines and started to adopt them in their flux de travail.

To reach photorealism, you need a ray-tracing renderer and a real-time 3D compositor like Reality Engine.

Reality Engine, le real-time node-based compositor, can key, track, render and composite photoreal graphics in real-time ⁠and has been natively built to take advantage of Moteur irréel features. This makes augmented reality graphics more photorealistic.

What’s more, the augmented reality graphics inside the Zero Density presentation are ray-traced, showcasing that cinematic-quality visuals are possible for ultra-realistic, live on-air graphics composited with live video. To reach this level of image fidelity, you need a ray-tracing renderer and a real-time 3D compositor like Reality Engine.

What an epic last day at @IBCShow! Just like the first days, it was still jam-packed at the Zero Density booth.

ZD team’s sincere thanks go to IBC Show 2022 for the exceptional organization, to numerous technology enthusiasts who paid a visit to our booth or followed us online. pic.twitter.com/NmjsLH8bgX

— ZeroDensity (@0Density) September 13, 2022

Reflections and Shadows

In augmented reality productions, the blending of the virtual and real can often be improved. As seen in the IBC Show demo, the presenter casts his reflections automatically on the AR graphics, creating an enhanced viewing experience for the audience. Adding a talent tracking product like Traxis talentS in the formula, as system knows the exact position of the talent inside the physical space, the reflections created on top of the AR graphics will be rendered at the correct position automatically. You can also unlock interactions with the graphics. Traxis talentS allows for automating triggers of AR graphics when the presenter moves to a specific area on the physical set.

Also, to help make your graphics look more natural and blend seamlessly with the studio, make sure to match the virtual shadows with the physical lights and ensure that your reflective objects always reflect the entire environment.

Blending

Last but not least of the capabilities, a unique feature of Reality Engine is compositing graphics with various blending modes similar to Photoshop, but in real-time. You can add bloom and lens flares additively blended over the lower third or apply blur effect on the background of the lower third on the incoming video signal.

Data-driven

AR data visualization delivers high value. It stands out from 2D content and secures more eyeballs on the show for longer. Once you have all your 3D assets designed and composited into your physical or virtual set environment, the last piece of the puzzle involves using a control application like RealityHub to feed those graphics with data. A 3D artist can download AR graphics, modify and have them ready to go in maybe a couple of hours but production won’t be functional until you can control it in real-time. That what makes a huge difference. With a control application like RealityHub, a designer can control the timing of animation and link real-time election, sports, and other data directly into the graphics.

For instance, RealityHub can use data from any source that can be used to feed the graphics, such as a local excel file or a live data feed, without using any additional plug-ins or writing a single line of code. RealityHub comes with a complete toolset to integrate any external data source and helps users to visualize complex information to the audience without the complexity in preparation. Make sure to adopt data driven graphics to your AR pipeline to add take advantage of easy-to-use systems and deliver an inspiring viewer experience.

Future of Broadcasting

Le immersive nature of augmented reality enables broadcasters to add a new dimension and dynamic to their shows, resulting in engaging audiences like never before. AR is proven to be no fad but the future of broadcasting. With augmented reality evolving rapidly and becoming more and more powerful every day, the possibilities become endless for broadcasters so they need to use an AR technology that can keep up. The Zero Density technology empowers you to deliver the most photorealistic AR graphics possible that meet your creative vision and targeted quality – now and in the future.

With augmented reality evolving rapidly and becoming more and more powerful every day, the possibilities become endless for broadcasters

With Moteur irréel 5 around the corner, we’re seeing more 3D artists starting to use Unreal Engine – it’s not just for games anymore. For anyone curious about learning about real-time graphics creation, I would recommend heading to the Jeux épiques website: they have plenty of free courses to get you started. I’d also recommend downloading the free community edition of our real-time compositing software, Reality Engine, and heading to our Zero Density Academy for some quick start videos to begin experimenting with.

Ahmet Oturgan

Business Development Manager – South Europe at Zero Density.