February 13, 2025
November 25, 2022
Huang, Zaiyu; Li, Hanhui; Xie, Zhenyu; Kampffmeyer, Michael; Cai, Qingling; Liang, Xiaodan.
In this paper, we target image-based person-to-person virtual try-on in the presenceof diverse poses and large viewpoint variations. Existing methods are restrictedin this setting as they estimate garment warping flows mainly based on 2D posesand appearance, which omits the geometric prior of the 3D human body shape.Moreover, current garment warping methods are confined to localized regions,which makes them ineffective in capturing long-range dependencies and results ininferior flows with artifacts. To tackle these issues, we present 3D-aware globalcorrespondences, which are reliable flows that jointly encode global semantic correlations, local deformations, and geometric priors of 3D human bodies. Particularly,given an image pair depicting the source and target person, (a) we first obtaintheir pose-aware and high-level representations via two encoders, and introduce acoarse-to-fine decoder with multiple refinement modules to predict the pixel-wiseglobal correspondence. (b) 3D parametric human models inferred from images areincorporated as priors to regularize the correspondence refinement process so thatour flows can be 3D-aware and better handle variations of pose and viewpoint. (c)Finally, an adversarial generator takes the garment warped by the 3D-aware flow,and the image of the target person as inputs, to synthesize the photo-realistic try-onresult. Extensive experiments on public benchmarks and our HardPose test setdemonstrate the superiority of our method against the SOTA try-on approaches.
Towards Hard-pose Virtual Try-on via 3D-aware Global Correspondence Learning
Huang, Zaiyu; Li, Hanhui; Xie, Zhenyu; Kampffmeyer, Michael; Cai, Qingling; Liang, Xiaodan.
Advances in Neural Information Processing Systems 2022 s. -
November 25, 2022
Huang, Zaiyu; Li, Hanhui; Xie, Zhenyu; Kampffmeyer, Michael; Cai, Qingling; Liang, Xiaodan.
Advances in Neural Information Processing Systems 2022 s. -
November 25, 2022