Application Guidelines

Application Guidelines

Application Deadline and Final Presentation

  • Application Submission Deadline: December 25th
  • Final Presentation and Awards Ceremony (Hybrid): February 8th, 8:00 JST

Overall Goal of the Challenge

The goal of this challenge is to realize an AI system that can answer questions with explanations.

Challenge Tasks

- There are 8 questions set for each of the two tasks, but there is no specific order to tackle them.
- Participants are welcome to work on the tasks and problems they are interested in and submit their completed works.
- Submissions are welcome even if they are only a part of a task or a partial effort.
  • - Task 1: Measure actions from a complete knowledge graph or scene graph
    • Answer Q1 to Q8 for each scenario provided in the scenario list.
    • The format for each answer is defined in JSON.
    • Posting the JSON to the answer API will calculate and return a score.
  • - Task 2: Measure actions from an incomplete knowledge graph or scene graph
    • Similar to Task 1, answer Q1 to Q8 for each scenario provided.
  • - Common Questions for Both Tasks
    • Q1: How many times did he enter each room?
    • Q2: How many times did he perform each action?
    • Q3: What action did he take after entering the kitchen?
    • Q4: What action did he take before entering the kitchen for the first time?
    • Q5: When, where, and what did he pick up?
    • Q6: What are he doing 10 seconds after the start?
    • Q7: Extract the relationship between objects at the initial state and after 10 seconds.
    • Q8: Extract changes in the state of objects from the initial state to 20 seconds later.

Conditions for Performing Tasks

  • - It is possible to independently expand the knowledge graph (e.g., adding common sense knowledge).
  • - If there are issues with the provided data, pull requests are acceptable.

Submissions

Submit a paper of at least 2 pages and no more than 8 pages. The paper should include the following information:

  • 1. The task you tackle.
  • 2. Explanation of the reasoning process.
  • 3. Explanation of the implementation.
  • 4. System can be under developemnt.
  • 5. A description of ideas only is acceptable.
  • 6. Interim results are acceptable.

Submission form: https://easychair.org/conferences/?conf=ikgrc2024

For more information, please visit the CFP.

Evaluation Criteria (Provisional)

  • - Common for all tasks: Accuracy rate, technical ingenuity, method of inference, expanded knowledge.
  • - Evaluation based on the paper submitted in advance and the presentation on the day.
  • * It is planned to conduct a peer review on the day of the event. At least one member from each team should participate in the review meeting (to be held on February 9, 2024). Presentations will be limited to teams whose papers have been accepted.