MAI@Home

As Embodied AI continues to evolve, understanding the spatiotemporal context of human actions in daily life is crucial for enabling robots to interact effectively with their environment. Datasets and benchmarks have been developed to facilitate research in this area, but many existing datasets suffer from limitations in data quality and annotation precision.

To address these challenges, we have created the Multimodal Dataset of Daily Life (MMDL) using a 3D VirtualHome-AIST simulator. MMDL offers a rich set of multimodal data, including:

• High-quality video data: Realistic simulations of everyday life activities.
• Event-centric spatiotemporal Knowledge Graphs: Detailed descriptions of events, locations, and objects in the environment.
• Consistent annotations: Mechanically generated annotations using a standardized vocabulary, ensuring consistency and reducing human error.

To further evaluate the capabilities of Embodied AI systems, we have also developed the Multimodal Question Answering Dataset of Daily Life (MMQADL). This dataset contains a variety of questions designed to test a system's ability to understand and reason about human behavior in complex real-world scenarios.

The goal of this challenge is to encourage the development of Embodied AI systems that can effectively comprehend and respond to human interactions in everyday settings. By participating in this challenge, researchers can benchmark their models against state-of-the-art techniques and contribute to advancing the field of Embodied AI.