Marathon Bib number recognition using deep learning

Abstract

Bib number recognition (BNR) from unstructured marathon images can be a challenging task. This is because the images captured at these events are very inconsistent since, they are often captured by multiple photographers, at various locations and times. This results in images containing different backgrounds, angles and illumination. The images often contain multiple participants in various poses, where the bib numbers can be obstructed by the participants themselves. The bib numbers are often printed on flexible paper and can easily be deformed which distorts the printed numbers. In this work we present a BNR system based on deep learning which is able to locate bib numbers in unstructured, complex marathon images. Using the segmented bib numbers the system then, recognizes the digits and finally outputs the bib numbers that it was able to detect in the image. The first stage consists of a fully Convolutional Neural Network (CNN) to segment the bib numbers while the second stage consists of a Convolutional Recurrent Neural Network (CRNN) used to recognize the detected numbers. The proposed method obtained an F1 score of 0.69 which outperformed existing methods.