Announcement
Oct 21, 2020 Winners' reports and presentation videos have been released. Winners presentation video
June 12, 2020 The content of the prize has been decided.

Purpose
While it is said that Moore’s Law, which has played an important role in the development of IT technology, is stalling, with the arrival of the IoT society, it is expected that the number of IoT devices that will be required to respond instantly at hand will increase. Therefore, the demand to realize innovative AI edge computing that uses AI to handle such processing is increasing. Since 2018, the Ministry of Economy, Trade and Industry (METI) and New Energy and Industrial Technology Development Organization (NEDO) has been holding the “AI Edge Contest”, which is to compete to solve problems with the intention of implementation using the real data for the purpose to discover excellent technologies and ideas and engineers who support them, and encourage new engineers to participate in this field.
The first contest was to compete the accuracy of algorithms with the theme of object detection and segmentation, the second one was on the theme of the implementation to FPGA, and this time for the third one is to compete recognition accuracy of the algorithm while paying attention to the model size and inference time with the theme of object tracking.

The 1st AI Edge Contest (Algorithm Contest (1))
The 2nd AI Edge Contest (Implementation Contest (1))
AI Edge Contest Official Website

Contest Outline
SubjectCreate an algorithm that detects a rectangular area in which objects are captured from the videos taken by a camera forward facing a vehicle and tracks the objects.
Data(Train/Test) videos of a vehicle front camera
(Train) Rectangular tag area labeled with objects, category and object ID
Identifying TargetCar, Pedestrian
Evaluation

For the models that clear the thresholds of the model size and inference time, determine the ranking based on the recognition accuracy of the prediction results.

Award, etc. 1st Prize: 500,000 yen + Google Cloud Platform Coupon (100,000 yen)
2nd Prize: 300,000 yen + Google Cloud Platform Coupon (50,000 yen)
3rd Prize: 100,000 yen + Google Cloud Platform Coupon (50,000 yen)

Subject
Allocate a rectangular area containing objects to be predicted as a bounding box = (x1, y1, x2, y2) to the video captured by the vehicle front camera, and give any unique object ID to the same object in each video. With allocations of multiple bounding boxes to each frame in the videos, the bounding box is represented by specifying four coordinates with the upper left corner as the origin (0,0), upper left coordinate of the object area (x1, y1), and the lower right coordinate (x2, y2).

However, the object to be evaluated (objects to be inferred) is limited to those that satisfy all of the following.
・There are two categories: “Car” and “Pedestrian”
・Objects with 3 or more frames in each video (Frames do not have to be continuous)
・Objects with a rectangle size of 1024 pix² or more

*Even if an object appears in three or more frames in the video, if there are less than 1024 pix² or more in three frames, the object including the size of 1024 pix² or more will not be evaluated.
*Please note that if an object that does not meet the conditions is detected, it is considered as a false detection.



Data Details

For TrainFor Test
RouteTokyo (Shibuya – Akihabara)
Time ZoneDay time
Resolution1936 x 1216
FPS5.0
Category TypesCar, Pedestrian, Truck, Signal, Signs, Bicycle, Motorbike, Bus, Svehicle, TrainCar, Pedestrian
Number of Videos25 videos74 videos
Length of an videos120 seconds30 seconds
Number of Frames per video600150

Travel Route

Posting Method
Post the source codes of the model itself and the inferencing part instead of the general prediction result file. After posting, creation of a prediction result file/evaluation of inference speed, and recognition accuracy are automatically processed.
Please refer to the following for the detail of the function and the posting method.
　・Detail of Function
　・Posting Method
* It takes time to reflect result of the evaluation on the leaderboard since the model will be executed.

Evaluation of Size/Inference Speed
Model size and inference time are evaluated according to the following flow.
　1. If the size of the posted model exceeds the threshold of 2GB, it will be ineligible for evaluation.
　2. The posted model will be executed in the following environment to measure the inference time and accuracy.
　　　 - OS: Debian GNU/Linux 10 (buster), version(10.3)
　　　 - GPU: Tesla V100
　　　 - CUDA Driver: Version 10.1.0
　　　 * See here for the Docker image of the executing environment and here for the Dockerfile.
　3. The threshold of inference time is 150 seconds / video (1 second / frame), and if it exceeds the threshold, it will be evaluated but not ranked. (It will be displayed on the leaderboard as “-”.)
　　* If it takes more than 3 hours and 30 minutes for including reading the model, an error will occur and it will be ineligible for evaluation.

　4. For the models that clear the model size and inference time thresholds, the ranking is determined based on the recognition accuracy of the prediction results. (The unit of inference time displayed on the leaderboard is seconds / video)



Evaluation of Recognition Accuracy
・ It is evaluated based on the evaluation function “MOTA (multiple object tracking accuracy)”
・ The maximum evaluation value is 1, and the higher the accuracy, the larger the value. It can be negative.
・ Please refer to the implementation example of the evaluation function which can be downloaded from the data page.

1. When the overlap (IoU) of the prediction area and the correct area (rectangle) exceeds 0.5 for all the frames in an video, it is determined that the area can be detected.
2. Calculate MOTA for each subjected prediction category based on the detected result.
3. Calculate the average of 2. of the all subjected prediction categories.
4. Calculate the average of 3. of the all videos for evaluation.



Reference: https://cvhci.anthropomatik.kit.edu/~stiefel/papers/ECCV2006WorkshopCameraReady.pdf

Decision for winner
1. Evaluations up to the last day of the contest (provisional evaluations) will be done based on the part of the evaluation data set, and evaluations after the contest (final evaluation) will be done on the remaining part of the evaluation data set.
The leaderboard automatically switches to the final evaluation at the end of the contest, and based on that, the final ranking is determined. Therefore, the ranking may change significantly during and after the contest.
*The inference speed does not change between the final evaluation value and provisional evaluation value
2. If the scores are the same, the participant who applied earlier will be ranked higher.
3. The office will contact the participants with the higher final ranking as the winning candidates.
At the time, the winning candidates will be asked whether to grant an open source license to the submission for the final review. If you do not grant this license, you will not be eligible to win. 4. Subject to no financial burden and infringement of rights for the office, the candidates must submit the followings.
・ Source code of the model (Excluding the inference parts posted at the time of submission, such as preprocessing and/or training. It is not required to submit the learned model.)
・ Source code of the manual (Specify as the preprocessing part, training part and inference part can be clear. If the data other than the provided data, leaned model, API, etc. are used, specify their source (location of the data, link to the paper, etc.)
・ Interpretation of the data, points for improvement, suggestions obtained from modeling, etc.
5. During the verification for reproducibility, if the winning candidates or the submitted models fall under any of the following, the qualification to win will be lost.
・ No response to the contact/request for procedure from the office within the specified deadline.
・ Conditions or rules for participation are not complied.
・ New video cannot be predicted.
6. Winners will be selected from those whose reproducibility is confirmed.

Comprehensive Ranking
・ This competition is subject to comprehensive rankings (scores and medals).

Use of System
・ One account is required for each participant. It is not acceptable to use multiple accounts by one participant and/or to share one account with multiple participants.
・ If you would like to participate in a team, please read here to create a team. (Deadline to create a team: 5/31. Up to 5 people including the representative.)
・ The maximum number of submission per day is five.

Handling of Information
・ Public disclosure of algorithms, ideas or other materials created to participate in this competition shall not be restricted, only if it is clearly stated that those are created to participate in this competition at the time of publication.
・ However, at the public disclosure, please specify it (including links) in the forum of this competition so that all participants can view it.

Use of Data
・ Use of the data other than provided data, and learned data shall not be restricted if the method allows verification of reproducibility for free. For example, the dataset with non-profit purpose, such as Waymo Open Dataset, can be used.
・ However, use of data provided by past AI Edge Contests is prohibited.
・ Modification of training data (including manual labeling and label rewriting) is allowed.
・ For API, the data that can be republished as an open source can be used.
・ Inference is performed on video-by-video basis. When tracking an object in videos, future information can be used if it is the data for test in the same video (tracking two-way). However use of the data in other videos is prohibited.

Host：Ministry of Economy, Trade and Industry / New Energy and Industrial Technology Development Organization (NEDO)
Co-host：Digital Media Professionals Inc. / SIGNATE Inc.
Support：IEEE Japan Office / Japan Electronics and Information Technology Industries Association (JEITA) / The Society of Instrument and Control Engineers (SICE) / The Institute of Electronics, Information and Communication Engineers (IEICE) / The Japan Society for Precision Engineering / Society of Automotive Engineers of Japan, Inc.
Sponsor：Google Japan LLC / Sompo Japan Nipponkoa Insurance Inc. / Digital Media Professionals Inc. / Panasonic Corporation / Dynamic Map Platform Co., Ltd. / Toyota Research Institute Advanced Development, Inc. / Sony Corporation / SoftBank Corp. / NEC Corporation / DENSO Corporation


Steering Committee Members
Shinpei Kato　Tokyo University
Kohei Ozaki　Algotechnica LLC
Atsuto Suyama　SB Drive Corp.
Naoki Suganuma　Kanazawa University
Yutaka Sato　National Institute of Advanced Industrial Science and Technology
Hiroaki Kitano　Sony Computer Science Laboratories
Hiroaki Matsumoto　Sony Semiconductor Solutions Corporation
Hironobu Tanba　SoftBank Corporation
Hironobu Fujiyoshi　Chubu University
Hiroki Nakahara　Tokyo Institute of Technology
Yoshiki Ninomiya　Nagoya University
Hirokazu Nishio　Mitou Foundation
Masaki Hiraga　Morpho, Inc.

Disclosure policy

As a general rule, in accordance with Article 4, Paragraph 1 of the terms of participation, diclosing any contents such as insights and deliverables transmitted through the information or data provided by our company in relation to this competition is not permitted, however, only after the completion of this competition and for non-commercial purposes, it will be possible to disclose the contents within the score of the table below
Model *1
Public
Analysis results *2
Public
Public : Posting to social media sites, blogs and source repositories, and citing to papers
Restricted : Using in a limited range from research, education to seminars, where many unspecified people cannot access
*1 Execution unit source code and learned models
*2 The insights obtained using the information and data provided, or the solutions including scripts and processed data such as summary statistics

Q. There was an obvious mistake in the label of the training data. What should I do?
A. It is possible to correct the learning data according to the rules, so please delete or correct it. We check all kinds of data, but it is difficult to detect and correct 100% of mistakes, so we take the present result as "correct" and analyze it. Thank you for your understanding.

Q. What happens to the ID when the same object is actually less than 1024 pix² in size, hidden behind another object, or left off the screen and then returned? 
A. Even if they are discontinuous, if they are actually the same object, they will be given the same ID.

Q. If there are the same objects in frames 1 to 3 at 2000, 2000 and 1000 pix² respectively, will only the objects in frames 1 and 2 be judged as labels? Or, since the number of frames is less than 3 as a result of being filtered by the rectangle size condition earlier, is it not all included in the label as a result?
A. All are not included in the correct answer label because they are filtered by the condition first. Even if an object appears in three or more frames in the video, if there are less than 1024 pix² or more in three frames, the object itself, including 1024 pix² or more, will not be evaluated.

Q. It seems that the code for calculating MOTA that can be downloaded on the data page does not include filtering of the three conditions to be inferred such as category, number of frames to appear, and size of rectangle.
A. When actually using the distributed MOTA code and verifying it in a local environment, please apply the conditions to the annotation that is the true value. If an object that does not meet the conditions is detected, it will be treated as a false detection. Therefore, when creating a post file, please post by yourself with the contents excluding the objects that satisfy the conditions removed.

Q. For "objects with more than 3 frames", is the first 2 frames that appear also considered in the judgment?
A. Yes. The judgment is made from the first frame.

Q. Do "objects with more than 3 frames" need to be continuous?
A. No. It does not have to be continuous.

▼ Special Contract on this Competition
In accordance with Article III “Prizes and Rights” of the special regulations established for this contest as described below, participants will either choose to make their own submission data, source code and other relevant items be licensed under an Open Source Initiative approved license (www.opensource.org) or not. This license imposes no restrictions on commercial use for licensed models, source code, and any results attained using said source code. Incidentally, the participants who will not choose this license will not be able to prize winner candidate, even though they can participate in the contest. In regard to posting and/or other publication on any social media site, blog or similar by the participant of information relating to any algorithm developed by said participant or any other materials produced for or related to this contest (including materials used for final submissions and/or final evaluations/inspections), no restrictions apply to posting and/or publication as long as the participant clearly explains that the posted/published items relate to this contest, and absolutely posts the positing and/or publication information (including the link URL to the pages) on the contest forum in order to let all the participants know it.


In order to participate in the Competition, you are required to agree to these Terms, in addition to the Terms of Use of SIGNATE.JP Site (hereinafter referred to as the “Terms of Use”). You should participate in the Competition after reading carefully and agreeing to these Terms. If you agree, these Terms, the matters that are added to these Terms as "additional matters", the Terms of Use and other terms and conditions that you have agreed to shall be binding on the relevant parties as integral documents.


Article 1. Definitions

１.For the purpose of these Terms, the following terms shall be defined as follows:

(１)"Site" means the website "SIGNATE (https://signate.jp)” on which the Competitions are posted.
(２)"Competition" means any competition on AI development or data analysis on the Site as held by the Host.
(３)"Host" is the host(s) of the Competitions. The Host may be SIGNATE, Inc. (hereinafter referred to as the “Company”) or the Company’s client companies, affiliated companies, schools or organizations, etc. (hereinafter referred to as the “Client(s)”).
(４)"Participant(s)" means the member(s) who participate in a Competition.
(５)"Submissions" means, collectively, the analysis and prediction results and reports, etc. as submitted in the Competition.
(６)"Final Submissions" means the Submissions that are specified by a Participant on the prescribed page in the Site by the time of completion of a Competition.
(７)"Winner Candidate" means the Participant who has received a notice from the Company that he/she is nominated as a winner candidate.
(８)"Submissions for Final Judgment" means the analysis and prediction model and learning data, etc. as submitted by a Winner Candidate pursuant to the instructions of the Company.
(９)"Final Judgment" means the acceptance inspection and judgment, including reproducibility verification, by the Company for the Final Submissions and Submissions for Final Judgment of a Winner Candidate.
(１０)"Winner" means the Winner Candidate who is informed by the Company that he/she has won a prize.
２.Unless otherwise defined in these Terms, the terms used in these Terms that are defined in the Terms of Use shall have the same meaning as defined in the Terms of Use.

Article 2. Competition

１.A member who desires to participate in a Competition shall be required to agree to these Terms and to satisfy the conditions for participation as specified in each such Competition. Any person who is not a member shall not participate in any Competition.
２.Participants shall participate in each Competition in the manner as advised by the Company and shall be obligated to comply with the rules as prescribed in each Competition.
３.Participants may submit the Submissions for the assignment of each Competition during the period of such Competition and submit a proposal on the method of solving the problem to the Host by the end of the period of the said Competition.
４.Participants may submit the Final Submissions in the form specified in each Competition by the time specified by the said Competition.
５.The Final Submissions as submitted shall be evaluated by the evaluation method as specified in each Competition and the final rank order shall be determined based on such evaluation.
６.Any Participant may, as a general rule, check the evaluation results of the Participant him/herself and each of the other Participants on the Site for the Submissions that may be evaluated quantitatively.
７.Participants shall be liable or otherwise responsible for their own Submissions, including their legality.
８.Participants shall not submit any Submissions that have no direct relationship to each Competition.
９.Unless otherwise provided for, Participants shall not directly communicate to, consult with, make a request to, solicit or take any other actions with the Host in respect of the matters related to a Competition during the period of the said Competition.
１０.Any Participant who has uncertainty or questions about any Competition shall make sure to contact the Company or its designee through the procedures prescribed by the Company as posted on the Site.
１１.The Company shall not be obligated to pay any remuneration or other consideration other than those prescribed in the following Article for any act of the Participants as prescribed in paragraphs hereof.

Article 3 Reward and Vesting of Rights

１.Unless otherwise provided for, any Participant shall satisfy the following requirements in order to be entitled to receive a reward in any Competition that offers a reward:

(１)To be a winner;
(２)To agree to transfer to the Host and the relevant transferee of rights in such Competition all transferable rights, such as copyrights, rights to obtain patents and know-how, etc. in and to all analysis and prediction results, reports, analysis and prediction model, algorithm, source code and documentations for the model reproducibility, etc., and the Submissions contained in the Final Submissions and Submissions for Final Judgment (including the rights as prescribed in Article 27 and Article 28 of the Copyright Act and the rights to obtain patents; hereinafter referred to as the "Rights");
(３)To agree that any relevant transferee of rights exclusively has the right to use the know-how contained in the Final Submissions and Submissions for Final Judgment for its own business and other purpose without any restriction;
(４)To agree not to exercise moral rights to the Rights against the relevant transferee of rights;
(５)To enter into an agreement for the transfer of the Rights with the relevant eligible transferee of rights, including the agreement to the matter in the preceding three (3) items and other reasonable provisions;
(６)To have the personal identity of such Participant verified by the Company.
(７)Not to breach any provision of these Terms and the Terms of Use.

２.Any Winner Candidate shall, after having received a notice from the Company that he/she is nominated as a winner candidate, submit the Submissions for Final Judgment on or before the designated date and communicate the matters requiring confirmation or response in relation to the Final Submissions and the Submissions for Final Judgment to the Company on or before the designated date, in accordance with the instructions of the Company. The Company shall carry out the final judgment based on such matters requiring confirmation or response. If the Company receives no confirmation or response satisfactory to the Company on or before the designated date, the Company may exclude such Winner Candidate from the subject of the final judgment.
３.If the Company considers that the Final Submissions or Submissions for Final Judgment need to be amended or modified, or there occur any additional matters requiring confirmation, in the course of the final judgment, any Winner Candidate shall take action or make response in relation to the matters that require amendment, etc. or the detailed information on the matters requiring confirmation, on or before the designated date in accordance with the instructions of the Company. If the Company receives no action or response satisfactory to the Company on or before the designated date, the Company may exclude such Winner Candidate from the final judgment.
４.The Company shall determine the Winner through the final judgment and inform the Winner to that effect.

Article 4 Confidentiality

１.Participants shall treat any information, data, or such contents as insights and deliverables transmitted through the service where they receive from the Company in relation to each Competition (hereinafter referred to as the "Company-Provided Information") as confidential information and shall not disclose the same to any third party and use the same for any purpose other than for such Competition and purpose specified by the Company separately; provided, however, that the confidential information shall not include any information that falls under any of the following items:

(１)Information that is known to the public at the time of the disclosure;
(２)Information that is already possessed by the Participant at the time of the disclosure (only in the case where such Participant may demonstrate such fact by reasonable means);
(３)Information that becomes known to the public without the fault of the Participant after the disclosure;
(４)Information that is independently developed by the Participant without reference to any information as disclosed (except for those Submissions of the person eligible for a prize which are evaluated); or
(５)Information that is rightfully disclosed by any third party having a right to do so without the obligations of confidentiality (only in the case where such Participant may demonstrate such fact by reasonable means).

２.Any Participant shall delete or return to the Company the Company-Provided Information immediately after the completion of each Competition.
３.Any Winner shall handle his/her Final Submissions and Submissions for Final Judgment in the same manner as prescribed in paragraph 1 hereof.
４.If there is any separate arrangement in relation to the confidential information in each Competition, the provisions of such arrangement shall prevail over the provisions of these Terms.
５.If any dispute occurs between the Host or other third party and the Company due to the breach by any Participant of the provisions of this Article and such other party makes any claim against the Company, such Participant shall compensate for any damage, loss, expenses (including, but not limited to, attorneys’ fees), lost profits and lost revenues, etc. incurred by the Company.
６.The provisions of this Article shall survive the termination of the relevant Competition or the Participant’s completion of the procedures for withdrawal from the service of the Company, with respect to the Company-Provided Information and the Winner’s Final Submissions and Submissions for Final Judgment for a period of five (5) years thereafter.

Article 5 Prohibited Acts of Participants

１．The Company shall prohibit Participants from engaging in any of the following acts in any Competition:

(１)An act of cracking, cheating, spoofing other misconduct;
(２)An act of directly communicating to, consulting with, making a request to, soliciting or responding to solicitation or other activities to other Participants or the Host (other than the Company) without the involvement of the Company;
(３)Any profitmaking activities using the Competition (including solicitation or scouting activities, and use for a third party in educational business, etc.) without the prior approval of the Company in writing or any other manner specified by the Company;
(４)Transfer, offering as collateral or other disposition of the status as a Participant or the rights or obligations as a Participant (except with the prior written consent of the Company); and
(５)Any other act in breach of the Terms of Use.

２.If the Company deems that a Participant engages in any of the prohibited acts as prescribed in the preceding paragraph, the Company may, without prior notice to the Participant, disqualify the Participant from the Competition in which the Participant participates, temporarily suspend the Participant from using the service of the Company, withdraw the Participant’s membership, claim damages from the Participant or take any other measures deemed necessary by the Company.

Article 6. Change, Discontinuation or Termination of Provision of Services under These Terms

１.The Company may change or temporarily suspend the services provided by the Company under these Terms without prior notice to the members.
２.Upon one (1) month prior notice to the members, the Company may suspend for a long period of time or terminate the services provided by the Company under these Terms.
３.The Company shall not be liable for any results or damage arising from the measures taken by the Company under this Article.

Article 7 Modification of Terms

１.The Company may modify, add or delete any provisions of these Terms from time to time without the approval of the members.

Enforced on April 1, 2018
Last updated on January 18, 2019