Implementation of a soccer match monitoring system for the “Computer Vision Challenge: Soccer Match Monitoring” at the IEEE DPVSA 2021. The team is formed by CS undergraduate students at Federal University Of Rio Grande Do Sul (UFRGS), Paulo Gamarra Lessa Pinto, Thiago Sotoriva Lermen and Gabriel Couto Domingues mentored by the Professor Claudio Rosito Jung, PhD.
In this project we have four main goals: team recognition, ball tracking, referee tracking and player tracking. For this task, we use classic computer vision techniques and machine learning.
The develpment process was using the Google Colab's environment. Therefore, our recommendations are to run the code using a Python Notebook inside the Colab's environment through the following link:
Using the link above you can read and execute the code, but if you want to run some tests or edit the code, you should copy the colab notebook to you own environment, for example Google Drive. Inside the Colab's envirment you can find other tips and specifications.
To train our machine learning models and validate the metrics we use an augmented DPVSA Dataset version. We used rotations between -11 and +11 degrees, horizontal flip, Hue between -37 and +37 degrees, brightness between -34% and +34%, exposure between -18% and +18%, blur up to 1px.
The original dataset is divided into different classes: ball, player_team_1, player_team_2, goalkeeper_team_1, goalkeeper_team_2, referee and outsider.To feed our model we replaced few base classes into a small subset. Our preprocessed dataset is splitted into ball, player, referee and outsider.
The following images are sample from the DPVSA dataset.
We divide the task into 2 different segments: object detection (including players, ball, referees and outsiders) and players clustering. First, we preprocess the dataset to obtain the data to train the model. After that, loading YOLOv5s model and training it from scratch, using the dataset classes mentioned in the last section. We perform object detection using the model to get all the bounding boxes predictions and the confidence score for each class.
As soon as the first segment is done, we have our object detection approach done. The next step is to divide the players by team. We use the clustering technique through K-means algorithm. We calculate the RGB average per channel for each predicted player bounding box and we classify this image in two different clusters: 'player_team_1' and 'player_team_2'.
To evaluate our model we focused on two different metrics and three different losses during both training and validation steps. The following image shows the results during the training and valitation during 100 epochs.
To compute the precision for each class, we computed the confusion matrix as follows:
