Can Apple’s HDR Augmented Reality Environments Solve Reflections For Neural Rendering?

Apple’s vigorous, long-term investment in Augmented Reality technologies is accelerating this year, with a new slate of developer tools to capture and convert real world objects into AR facets, and a growing industry conviction that dedicated AR eyewear is coming to support the immersive experiences that this blizzard of R&D can enable.

Among a tranche of new information on Apple’s efforts in Augmented Reality, a new paper from the company’s computer vision research division reveals a method for using 360-degree panoramic high dynamic range (HDR) images to provide scene-specific reflections and lighting for objects that are superimposed into augmented reality scenes.

Entitled HDR Environment Map Estimation for Real-Time Augmented Reality, the paper, by Apple Computer Vision Research Engineer Gowri Somanath and Senior Machine Learning Manager Daniel Kurz, proposes the dynamic creation of real-time HDR environments via a convolutional neural network (CNN) running in a mobile processing environment. The result is that reflective objects can literally mirror novel, unseen environments on demand:

In Apple’s new AR object generation workflow, a pressure cooker is instanced by photogrammetry complete with its ambient environment, leading to convincing reflections that aren’t ‘baked’ into the texture. Source: https://docs-assets.developer.apple.com/

Leave a Comment