Monitoring Activities from Multiple Video Streams: Establishing a Common Coordinate Frame

By Lily Lee, Raquel A. Romano, Gideon P. Stein. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000

411 citations recorded by the source corpus.

Monitoring of large sites requires coordination between multiple cameras, which in turn requires methods for relating events between distributed cameras. This paper tackles the problem of automatic external calibration of multiple cameras in an extended scene, that is, full recovery of their 3D relative positions and orientations. Because the cameras are placed far apart, brightness or proximity constraints cannot be used to match static features, so we instead apply planar geometric constraints to moving objects tracked throughout the scene. By robustly matching and fitting tracked objects to a planar model, we align the scene's ground plane across multiple views and decompose the planar alignment matrix to recover the 3D relative camera and ground plane positions. We demonstrate this technique in both a controlled lab setting where we test the effects of errors in the intrinsic camera parameters, and in an uncontrolled, outdoor setting. In the latter, we do not assume synchronized cameras and we show that enforcing geometric constraints enables us to align the tracking data in time. In spite of noise in the intrinsic camera parameters and in the image data, the system successfully transforms multiple views of the scene's ground plane to an overhead view and recovers the relative 3D camera and ground plane positions.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/5c2e57744b643871d28b4a48f3ffabc5a44aa370

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Tracking across multiple cameras with disjoint views (2003, background): “[14] proposed an approach for tracking in cameras with overlapping FOV’s that does not require calibration.”
Consistent Labeling of Tracked Objects in Multiple Cameras with Overlapping Fields of View (2003, methodology): “S. Khan is with the Department of Computer Science, Lahore University of Management Sciences, Sector U, D.H.A., Lahore, 54792 Pakistan.”
Multi-camera activity correlation analysis (2009, background): “Discovering and modelling such activity correlations among multiple camera views from data directly can facilitate person re-identification across disjoint camera views [2, 3, 5–7, 14] and global activity analysis [9, 18, 21].”
A survey on visual surveillance of object motion and behaviors (2004, background): “Stein and Lee et al. [38], [39] use the motion trajectory and the ground plane constraint to determine the projection transformation matrix, and then such matrix is decomposed to obtain the extrinsic parameters of the camera.”
Object tracking: A survey (2006, methodology): “2001; Cai and Aggarwal 1999] or computed automatically [Lee et al. 2000; Khan and Shah 2003] from the”
Algorithms for cooperative multisensor surveillance (2001, background): “image pixels and terrain locations is a simple 2-D homography [9], [49], [74], [75].”
Learning Patterns of Activity Using Real-Time Tracking (2000, methodology): “Calibration of cameras and extraction of ground plane information are covered separately in [ 18 ].”
Modeling inter-camera space-time and appearance relationships for tracking across non-overlapping views (2008, background): “Thus, the color distribution of an object can be fairly different when viewed from two different cameras.”
Intelligent multi-camera video surveillance: A review (2013, background): “Lee et al. (2000) track objects simultaneously in partially overlapped camera views and use the object centroids as potential point correspondences to recover the homography between two camera views with planar geometric constraints on the moving objects.”