An iterative integrated framework for thermal-visible image registration, sensor fusion, and people tracking for video surveillance applications

By Atousa Torabi, Guillaume Massé, Guillaume-Alexandre Bilodeau. 2012

165 citations recorded by the source corpus.

In this work, we propose a new integrated framework that addresses the problems of thermal-visible video registration, sensor fusion, and people tracking for far-range videos. The video registration is based on a RANSAC trajectory-to-trajectory matching, which estimates an affine transformation matrix that maximizes the overlapping of thermal and visible foreground pixels. Sensor fusion uses the aligned images to compute sum-rule silhouettes, and then constructs thermal-visible object models. Finally, multiple object tracking uses blobs constructed in sensor fusion to output the trajectories. Results demonstrate the advantage of our proposed framework in obtaining better results for both image registration and tracking than separate image registration and tracking methods.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Michael Ying Yang
2. Hui Cheng
3. Dimitris N. Metaxas
4. Margrit Betke
5. Liang Lin
6. Shaoting Zhang
7. Yukyung Choi
8. Chenglong Li

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Ming-Hsuan Yang
4. In-So Kweon
5. Liang Lin
6. Xiaogang Wang
7. Alan L. Yuille
8. Trevor Darrell

In Media Technology

1. Michael Ying Yang
2. Dimitris N. Metaxas
3. Liang Lin
4. Shaoting Zhang
5. Jingjing Liu
6. Shu Wang
7. In-So Kweon
8. Soonmin Hwang

Citations in the corpus

• RGB-T Object Tracking: Benchmark and Baseline (2018, methodology): “There are several video benchmark datasets designed for RGB-T tracking purpose such as OSU-CT [4], LITIV [5] and GTOT [6].”
• KAIST Multi-Spectral Day/Night Data Set for Autonomous and Assisted Driving (2018, methodology): “4, and they have been used as night vision sensors in many academic and industrial applications [5]–[7].”
• Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection (2018, background): “To overcome the aforementioned limita tions, many research works have been focused on the development of multispectral pedestrian detection solutions to fa cilit te robust human target detection for around-the-clock applic ation [22, 21, 34, 28, 16, 13].”
• LasHeR: A Large-Scale High-Diversity Benchmark for RGBT Tracking (2021, background): “Early works develop some smallscale datasets including OSU-CT [5] and LITIV [6], which contain six and nine RGBT videos respectively.”
• A Thermal Infrared Video Benchmark for Visual Analysis (2014, methodology): “A few of thermal infrared dataset have been published in the past, e.g., the OTCBVS Benchmark 1, the LITIV Thermal-Visible Registration Dataset [3], the AIC ThermalVisible Night-time Dataset [4], and the ASL Thermal Infrared Dataset [5] (Table I).”
• Multispectral Deep Neural Networks for Pedestrian Detection (2016, background): “Since ambient lighting has less effect on thermal imaging, thermal cameras are widely used in human tracking [41], face recognition [31], and activity recognition [14] for its robustness.”
• Learning Collaborative Sparse Representation for Grayscale-Thermal Tracking (2016, background): “In particular, OSU Color-Thermal [5] and LITIV [6], [7] datasets contain 6, 9 and 4 grayscale-thermal video sequences, respectively.”