IDD-X: A Multi-View Dataset for Ego-relative Important Object Localization and Explanation in Dense and Unstructured Traffic

ICRA 2024

Chirag Parikh1, Rohit Saluja2, C. V. Jawahar1, Ravi Kiran Sarvadevabhatla1
1IIIT Hyderabad, 2IIT Mandi
arXiv Video Poster Code Data

Sample Annotated Driving Scenarios

Abstract

Intelligent vehicle systems require a deep understanding of the interplay between road conditions, surrounding entities, and the ego vehicle's driving behavior for safe and efficient navigation. This is particularly critical in developing countries where traffic situations are often dense and unstructured with heterogeneous road occupants. Existing datasets, predominantly geared towards structured and sparse traffic scenarios, fall short of capturing the complexity of driving in such environments.

To fill this gap, we present IDD-X, a large-scale dual-view driving video dataset. With 697K bounding boxes, 9K important object tracks, and 1-12 objects per video, IDD-X offers comprehensive ego-relative annotations for multiple important road objects covering 10 categories and 19 explanation label categories. The dataset also incorporates rearview information to provide a more complete representation of the driving environment.

We also introduce custom-designed deep networks aimed at multiple important object localization and per-object explanation prediction. Overall, our dataset and introduced prediction models form the foundation for studying how road conditions and surrounding entities affect driving behavior in complex traffic situations.

Video

Existing Datasets Comparisons

IDDX Comparisons

References: DRAMA [3]; H3D [1]; BDD-OIA [4]; OIE [2]; HDD [6]; BDD-X [5]; METEOR [7].

Our Dataset

Important Object Explanations

The dataset contains 19 Ego-relative Explanation Categories which can broadly be grouped as: (I) Passive influence on ego-vehicle's driving decision, and (II) Ego-relative maneuvering styles of Important Objects.

IDDX Explanations

Statistics

The explanations for important objects follow a heavy-tail distribution. Their average duration estimated using the object's track length is also reported.

IDDX Statistics

Our Approach

We introduce custom-designed deep networks for (I) Important Object Track Identification, and (III) Important Object Explanation Prediction with (II) Ego-Vehicle's Driving Behavior Recognition.

IDDX Approach

Results

Established new benchmarks using our custom-designed deep networks for the proposed tasks (I), (II), and (III) in dense and unstructured traffic scenarios.

IDDX Results

Download the Dataset

This dataset corresponds to the paper, "IDD-X: A Multi-View Dataset for Ego-relative Important Object Localization and Explanation in Dense and Unstructured Traffic". Use this link to sign up and make the download request.

BibTeX

@misc{parikh2024iddx,
      title={IDD-X: A Multi-View Dataset for Ego-relative Important Object Localization and Explanation in Dense and Unstructured Traffic}, 
      author={Chirag Parikh and Rohit Saluja and C. V. Jawahar and Ravi Kiran Sarvadevabhatla},
      year={2024},
      eprint={2404.08561},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}