IntentNet: Learning to Predict Intention from Raw Sensor Data

By Sergio Casas, Wenjie Luo, Raquel Urtasun. Conference on Robot Learning, 2018

420 citations recorded by the source corpus.

In order to plan a safe maneuver, self-driving vehicles need to understand the intent of other traffic participants. We define intent as a combination of discrete high-level behaviors as well as continuous trajectories describing future motion. In this paper, we develop a one-stage detector and forecaster that exploits both 3D point clouds produced by a LiDAR sensor as well as dynamic maps of the environment. Our multi-task model achieves better accuracy than the respective separate modules while saving computation, which is critical to reducing reaction time in self-driving applications.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, Social Psychology

View Paper: https://www.semanticscholar.org/paper/4af04c80dc28af634e0552f4e74093fb363fa377

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Social Psychology

Citations in the corpus

3D Multi-Object Tracking: A Baseline and New Evaluation Metrics (2019, background): “ywords: 3D multi-object tracking, real-time, evaluation metrics 1 Introduction Multi-object tracking (MOT) is an essential component technology for many vision applications such as autonomous driving [1,2,3], robot collision prediction [4,5] and video face alignment [6, 7,8]. Due to the signiﬁcant a”
MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction (2019, methodology): “As in [2, 6], we assume that an industry-grade perception system provides sufﬁciently accurate poses and tracks for all nearby agents, including vehicles, pedestrians, and cyclists.”
Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data (2020, background): “The few methods which do include such considerations are closed-source, tightly integrated with a specific robotic platform, or trained on private data [50,8,20,7].”
Multiple Futures Prediction (2019, background): “The success of machine learning and deep learning ushered in a variety of data-driven recurrent neural network (RNN) based methods [24, 7, 13, 26, 16, 2].”
TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving (2022, background): “Most sensor fusion works consider perception tasks, e.g. object detection [14], [15], [16], [18], [19], [20], [21], [22], [23], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60] and motion forecasting [24], [25], [26], [27], [28], [29], [30], [49], [61], [62].”
MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction (2021, background): “By rasterizing all world state in a top-down orthographic view, MultiPath and others [10, 14, 27, 30, 33, 37, 46, 52] leverage powerful, established CNN architectures like ResNet [25], which offer solutions to the heterogeneous interrelated input space: the heterogeneous world state is mapped to a…”
Argoverse: 3D Tracking and Forecasting With Rich Maps (2019, methodology): “IntentNet [6] extends the joint detection and prediction approach of Luo et al.”
Query-Centric Trajectory Prediction (2023, background): “Early work rasterizes world states as multichannel images and employs classic convolutional neural networks for learning [5,6,10,21].”
3D Object Detection for Autonomous Driving: A Comprehensive Survey (2022, background): “Point cloud and map samples are from (Casas et al., 2018)”
Motion Transformer with Global Intention Localization and Local Movement Refinement (2022, background): “IntentNet [7] considers intention prediction as a classification with 8 high level actions, while [29] proposes a region-based to provide trajectory-specific feature for motion decoder network.”
TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors (2021, methodology): “First, we rasterize the HD mapM and use a CNN-based perception backbone network inspired by [42, 13] to extract rich geometrical features M̃, shown in Figure 3 (a).”
End-To-End Interpretable Neural Motion Planner (2019, methodology): “Similar to [5], we rasterize the map to form an M channels tensor, where each channel represents a different map element, including road, intersections, lanes, lane boundaries, traffic lights, etc.”
CoverNet: Multimodal Behavior Prediction Using Trajectory Sets (2019, background): “Numerous papers are evaluated purely on internal datasets [13, 35, 6, 28], as common public datasets are either relatively small [16], focused on highway driving [9], or are tangen-tially related to driving [32].”
VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation (2020, methodology): “Our baseline uses a ConvNet to encode the rasterized images, whose architecture is comparable to IntentNet [5]: we use a ResNet-18 [14] as the ConvNet backbone.”
MP3: A Unified Model to Map, Perceive, Predict and Plan (2021, methodology): “Following [9], we concatenate height and time along the channel dimension to avoid using 3D convolutions or a recurrent model, thus saving memory and computation.”
V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction (2020, methodology): “ algorithm performance has been further improved by building good multimodal distributions [4,6,12,19] that capture diverse actor behaviour and by modelling actor interactions [3,25,36,37]. Recently, [5, 31] propose approaches that perform joint perception and motion forecasting, dubbed perception a”
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset (2021, neutral): “[1, 3, 7, 5, 8, 12, 11, 14, 17, 20, 22, 25, 39].”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “[121] leverage HD maps to provide prior knowledge of the road topology.”