Panoptic Studio: A Massively Multiview System for Social Motion Capture

By Hanbyul Joo, Hao Liu, Lei Tan, Lin Gui, Bart C. Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, Yaser Sheikh. IEEE International Conference on Computer Vision, 2015

661 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/586eb9f2eb5d963ef95c266eab2723bd8d7911a3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Hand Keypoint Detection in Single Images Using Multiview Bootstrapping (2017, methodology): “For multiview bootstrapping, we use images from the Panoptic Studio dataset [8].”
You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions (2019, methodology): “We validate our You2Me ego-pose approach on two forms of ground-truth capture—from Kinect sensors and a Panoptic Studio [28]—on video data spanning 10 subjects and several interaction domains (conversation, sports, hand games, and ball tossing).”
PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction (2020, methodology): “To capture detailed motions of multiple interacting characters, more than six hundred cameras have been used to overcome the self-occlusion challenges [25], [26].”
H2O: Two Hands Manipulating Objects for First Person Interaction Recognition (2021, methodology): “A few datasets collect hand pose ground truth, acquired in an automated or semi-automated way (Panoptic [37, 38], FreiHand [99], Interhand [55]).”
Monocular Total Capture: Posing Face, Body, and Hands in the Wild (2018, methodology): “Newly Captured Total Motion Dataset: We use the Panoptic Studio [24, 25] to capture a new dataset for 3D body and hand pose in a markerless way [26].”
Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies (2018, methodology): “For experiments, we use the dataset captured in the CMU Panoptic Studio [26].”
Human Motion Diffusion as a Generative Prior (2023, methodology): “Motion data is typically either acquired by elaborate motion capture settings [Joo et al. 2015] or crafted by artists [Adobe Systems Inc. 2021].”
Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose (2020, methodology): “ above methods, which are image-based, in that it uses the 2D human pose as an input. The proposed system can benet from the data with 3D annotations, which are captured from controlled environments [21,25], without the appearance domain gap issue. GraphCNN for mesh processing. Recently, many method”
Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision (2016, methodology): “Additionally, we qualitatively observe the performance on LSP [30] and the CMU Panoptic [32] datasets, demonstrating robustness to general scenes.”
FrankMocap: Fast Monocular 3D Hand and Body Motion Capture by Regression and Integration (2020, background): “MTC. Monocular Total Capture [55] is a dataset captured by Panoptic Studio [16, 18] in a multi-view setup with 30 HD cameras.”
3D Hand Shape and Pose From Images in the Wild (2019, background): “The PANOPTIC dataset [44] contains hands in various poses observed from multiple views in the Panoptic studio [16].”
MotionDiffuse: Text-Driven Human Motion Generation With Diffusion Model (2022, background): “A common form of datasets containing videos of human subjects are recorded with annotations such as 2D keypoints [49], [50], 3D keypoints [6], [51], [52], [53], [54] and statistical model parameters [55], [56], [57], [58], [59].”
Panoptic Studio: A Massively Multiview System for Social Interaction Capture (2016, methodology): “The major failures of [56] occur on the texture-less leg parts, or fast motion withmotion blur when the trajectory stream is Fig.”
Learning Monocular 3D Human Pose Estimation from Multi-view Images (2018, background): “In a controlled studio environment, marker-suits [9] and marker-less motion capture systems [16, 11, 19] can be used to estimate the pose automatically.”
Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis (2023, methodology): “We perform experiments using synchronized multi-view video (27 training cameras, 4 testing cameras) from the CMU Panoptic Studio dataset [15].”
Single-Shot Multi-person 3D Pose Estimation from Monocular RGB (2017, methodology): “[24] using a multi-camera studio setup but background diversity remains limited.”
Monocular, One-stage, Regression of Multiple 3D People (2020, methodology): “This pixel-level body-joint representation enables their impressive performance in crowded scenes [7, 8, 40].”
Expressive Body Capture: 3D Hands, Face, and Body From a Single Image (2019, background): “3D models [7, 20, 24, 30, 35, 46, 54, 66, 72].”
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion (2019, background): “Modern systems such as the Panoptic Studio [14] can align 3D reconstructions without markers, but require complex specialized hardware, making them unsuitable for use outside a specialized laboratory.”