Search code, repositories, users, issues, pull requests...
Currently, we release two types of models: text-conditioned relighting model and background-conditioned model. Both types take foreground images as inputs.
(Note that the "Lighting Preference" are just initial latents - eg., if the Lighting Preference is "Left" then initial latent is left white right black.)
The background conditioned model does not require careful prompting. One can just use simple prompts like "handsome man, cinematic lighting".
Using the above light stage as an example, the two images from the "appearance mixture" and "light source mixture" are consistent (mathematically equivalent in HDR space, ideally).
As a result, the model is able to produce highly consistent relight - so consistent that different relightings can even be merged as normal maps! Despite the fact that the models are latent diffusion.
From left to right are inputs, model outputs relighting, devided shadow image, and merged normal maps. Note that the model is not trained with any normal map data. This normal estimation comes from the consistency of relighting.
