Recently, Style-GAN enabled high-resolution artistic image creation through transfer learning. However, current strategies learn only an overall translation of the distribution, which are unable to perform similative-based style transfer.
Portrait generation – artistic interpretation. Image credits: Tumisu via Pixabay, free license
A recent paper on arXiv.org examines the problem of example-based portrait style transfer, a problem that aims to transfer the style of an exemplary artistic portrait to a target face.
To realize the effective modeling and control of dual styles, the researchers propose a novel dual-stylegain. The method retains an internal style path of StyleGAN to control the style of the parent domain, but also an external style path to model and control the style of the target extended domain.
The novel formulation enables high-quality and high-resolution pastiche and provides flexible and varied control over both color styles and complex structural styles.
Recent studies on StyleGAN show high performance on artistic image creation by transfer learning with limited data. In this paper, we explore the more challenging example-based high-resolution portrait style transfer by introducing a novel Dual StyleGain with flexible control of the dual styles of the original face domain and the extended artistic portrait domain. Unlike StyleGAN, DualStyleGAN provides a natural way of style transfer by drawing the content and style of a picture with an internal style path and a new external style path, respectively. The delicately designed exterior style path enables our model to sequentially modify both color and complex structural styles to accurately affix the style instance. In addition, a new progressive fine-tuning scheme has been introduced to smoothly transform the model’s generative space into the target domain, even with the above modifications on the network architecture. Experiments demonstrate the superiority of Dual StyleGain over cutting edge methods in high quality portrait style transfer and flexible style control.
Research Paper: Yang, S., Jiang, L., Liu, Z., and Chang Loy, C., “Pastic Master: Exemplar-based High-Resolution Portrait Style Transfer”, 2022. Article link: https://arxiv. org/abs/2203.13248
Project Page: https://www.mmlab-ntu.com/project/dualstylegan/
