Convolutional neural network for ingredient detection

Abstract

Detecting ingredients from ready-made food dishes is an exceptional way of monitoring one’s daily food intake. It lays the foundation for solving other culinary, vision-related problems throughout the whole food supply chain. We attempt to design a convolutional neural network which conducts classification on a Chinese cuisine dataset, VIREO-251. Such a task is rather complex, given that ingredients lie within a multi-labelled setting, and even more so when considering the highly versatile shapes that certain ingredients possess. An 18-layer ResNet model is trained to classify a subset of 15 ingredients, using a threshold probability to determine their presence. The system achieves an F1-score of 15.1%, as well as top-1 and top-2 scores of 14.8% and 25.9% respectively.