GazeDirector: Fully Articulated Eye Gaze Redirection in Video

By Erroll Wood, Tadas Baltrusaitis, Louis-philippe Morency, Peter Robinson, Andreas Bulling. 2017

59 citations recorded by the source corpus.

We present GazeDirector, a new approach for eye gaze redirection that uses model‐fitting. Our method first tracks the eyes by fitting a multi‐part eye region model to video frames using analysis‐by‐synthesis, thereby recovering eye region shape, texture, pose, and gaze simultaneously. It then redirects gaze by 1) warping the eyelids from the original image using a model‐derived flow field, and 2) rendering and compositing synthesized 3D eyeballs onto the output image in a photorealistic manner. GazeDirector allows us to change where people are looking without person‐specific training data, and with full articulation, i.e. we can precisely specify new gaze directions in 3D. Quantitatively, we evaluate both model‐fitting and gaze synthesis, with experiments for gaze estimation and redirection on the Columbia gaze dataset. Qualitatively, we compare GazeDirector against recent work on gaze redirection, showing better results especially for large redirection angles. Finally, we demonstrate gaze redirection on YouTube videos by introducing new 3D gaze targets and by manipulating visual behavior.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Yujiu Yang
2. Weihao Xia
3. Ming-Hsuan Yang
4. Jing-Hao Xue
5. Luc Van Gool
6. Yulun Zhang
7. Bolei Zhou
8. Xiaogang Wang

In Radiology, Nuclear Medicine and Imaging

1. Zhe He
2. Ming-Hsuan Yang
3. Adrian Spurr
4. Xucong Zhang
5. Otmar Hilliges

In Human-Computer Interaction

1. Jing-Hao Xue
2. Yujiu Yang
3. Weihao Xia
4. Wensen Feng
5. Otmar Hilliges
6. Zhe He
7. Bolei Zhou
8. Ming-Hsuan Yang

Citations in the corpus

• GAN Inversion: A Survey (2021, background): “, gaze redirection [7], [231], [232], [233], relighting [234], [235], [236] and continuous view control [237].”
• Photo-Realistic Monocular Gaze Redirection Using Generative Adversarial Networks (2019, methodology): “Therefore, we did not compare GazeDirector in our paper.”
• Controllable Continuous Gaze Redirection (2020, background): “Some traditional approaches are based on 3D modeling [2, 36].”