It is difficult to take a portrait picture under direct sunlight. Professional photographers use elaborate lighting, but casual users must rely on alternative solutions.
Portrait of a man. Image credits: Reza Sad, CC-BY-SA-4.0 via Wikimedia
A recent paper published on arXiv.org proposes a method to enable high-quality, physically grounded portraits without additional hardware.
The proposed capture setup requires the camera to be rotated, allowing the face to be viewed under different sunlight angles. The method then reconstructs geometry, materials, camera posture and lighting information. This information is used to realistically represent the subject in invisible light conditions.
Complex details that do not explicitly model the geometry, such as self-inflicted cast shadows, are also rendered realistically. In addition, the method can be used to present the person in an entirely new environment, or add freckles, makeup, or stickers that realistically interact with visible lighting.
Outdoor portrait photos are often affected by harsh shadows cast under direct sunlight. To solve this, one can use post-capture lighting manipulation techniques, but these methods require complex hardware (eg, a light stage) to capture each individual, or rely on image-based priors. and thus many fail to reconstruct the subtle face. Details that vary from person to person. In this paper, we present Sunstage, a system for precise, individually tailored and light reconstructions of facial geometry and reflectance that can be used for general portrait relativization with cast shadows. Our method requires the user to simply capture a selfie video, while walking outside, and use the different angles between the Sun and the face as constraints in the combined reconstruction of facial geometry, reflectance properties, and lighting parameters. does. In addition to relighting, we show that our reconstruction can be used for applications such as reflection editing and visualization of synthesis. The results and interactive demo are available at this https URL.
Research Article: Wang, Y., Holinsky, A., Zhang, X., & Zhang, X. C., “Sunstage: Portrait Reconstruction and Relighting Using the Sun as a Light Stage”, 2022. Link: https://arxiv.org/abs/ 2204.03648
