Hello! My name is... Buffy'' -- Automatic Naming of Characters in TV Video
By Mark Everingham, Josef Sivic, Andrew Zisserman. 2006
689 citations recorded by the source corpus.
We investigate the problem of automatically labelling appearances of characters in TV or film material. This is tremendously challenging due to the huge variation in imaged appearance of each character and the weakness and ambiguity of available annotation. However, we demonstrate that high precision can be achieved by combining multiple sources of information, both visual and textual. The principal novelties that we introduce are: (i) automatic generation of time stamped character annotation by aligning subtitles and transcripts; (ii) strengthening the supervisory information by identifying when characters are speaking; (iii) using complementary cues of face matching and clothing matching to propose common annotations for face tracks. Results are presented on episodes of the TV series “Buffy the Vampire Slayer”.
Subfields: Computer Vision and Pattern Recognition, Information Extraction
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Xiaogang Wang
- Bernt Schiele
- Andrew Zisserman
- Jitendra Malik
- Cordelia Schmid
In Information Extraction
- Ivan Laptev
- Tinne Tuytelaars
- Punarjay Chakravarty
- Lior Wolf
- Juergen Gall
- Tal Hassner
- Itay Maoz
- B. Taskar
Citations in the corpus
- A dataset for Movie Description (2015, background): “Their corpus is based on the original sources to create the DVS and contains different kinds of artifacts not present in actual description, such as dialogs and production notes.”
- Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition (2019, background): “[8, 21, 23, 47] analyzed movie scripts for automatic annotation of human actions for recognizing and localizing actions and actors.”
- Person Recognition in Personal Photo Collections (2015, background): “Other related tasks are, for instance, face clustering [9, 33], finding important people [30], or associating names in text to faces in images [13, 14].”
- A Survey on Human Motion Analysis from Depth Data (2013, methodology): “of the facial features configuration like pictorial structures [78, 79] or Active Shape Models [80] can be used.”
- Movie/Script: Alignment and Parsing of Video and Text Transcription (2008, methodology): “Following [14], we detect onscreen speakers as follows: 1) locate mouth for each face track using a mouth detector based on Viola-Jones, 2) compute a mouth motion score based on the normalized cross correlation between consecutive windows of the mouth track, averaged over temporal segments correspon”
- Cross-Modal Supervision for Learning Active Speaker Detection in Video (2016, methodology): “Traditionally, visual active speaker detection has been done using lip motion detection [1,2,3,4].”