Virtual try-all: Visualizing any product in any personal setting
A way for online shoppers to virtually try out products is a sought-after technology that can create a more immersive shopping experience. Examples include realistically draping clothes on an image of the shopper or inserting pieces of furniture into images of the shopper’s living space.
In the clothing category, this problem is traditionally known as virtual try-on; we call the more general problem, which targets any category of product in any personal setting, the virtual try-all problem.
In a paper we recently posted in arXiv, we presented a solution to the virtual-try-all problem called Diffuse-to-Choose (DTC). Diffuse-to-Choose is a novel generative-AI model that allows users to seamlessly insert any product at any location in any scene.
The customer starts with a personal scene image and a product and draws a mask in the scene to tell the model where to insert the object. The model then integrates the item into the scene, with realistic angles, lighting, shadows, and so on. If necessary, the model infers new perspectives on the item, and it preserves the item’s fine-grained visual-identity details.
