Can deep learning models recognize chilean diet?
Keywords:
Food recognition, Food dataset, Deep Learning, Chilean dietAbstract
The emergence of deep learning models has made it possible to address several real-life problems and, in particular, those in which computer vision plays a key role. In this sense, the food recognition task from images is one of the beneficiaries of this machine learning method. Its importance lies in its usefulness to become aware of the food eaten and in this way help us to lead a healthy lifestyle. In recent years, food image recognition has gained great prominence in the literature, providing novel models and datasets to address it. However, public data generally correspond to American, Asian, and European foods, therefore the methods developed cannot be directly applied to the Chilean diet. In this article we will publish a new dataset to recognize the foods present in the Chilean diet. In addition, we will perform a comparison with public popular food datasets to analyze the similarity in the dishes of the proposed dataset with respect to the exciting ones in the literature. Moreover, we will establish a baseline using the state-of-the-arts Convolutional Neural Network architectures and the novel Swin Transformer approach.
Downloads
References
M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in Int. Conf. on Mach. Learn., pp. 6105–6114,PMLR, 2019.
L. Liu, W. Ouyang, X. Wang, P. Fieguth, J. Chen, X. Liu, andM. Pietikäinen, “Deep learning for generic object detection: A survey,”Int. J. of computer vision, vol. 128, no. 2, pp. 261–318, 2020.
A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, andA. A. Bharath, “Generative adversarial networks: An overview,”IEEESignal Processing Magazine, vol. 35, no. 1, pp. 53–65, 2018.
I. Kyrou, H. S. Randeva, C. Tsigos, G. Kaltsas, and M. O. Weickert,“Clinical problems caused by obesity,”Endotext [Internet], 2018.
“World obesity, obesity: missing the 2025 global targets – trends, costs and country reports,” 2020.
T. Kelly, W. Yang, C.-S. Chen, K. Reynolds, and J. He, “Global burden of obesity in 2005 and projections to 2030,”Int. J. of obesity, vol. 32,no. 9, pp. 1431–1437, 2008.
R. D. Telford, “Low physical activity and obesity: causes of chronic disease or simply predictors?,”Medicine & Science in Sports & Exercise,vol. 39, no. 8, pp. 1233–1240, 2007.
B. M. Kuehn, “More severe obesity leads to more severe covid-19 in study,”JAMA, vol. 325, no. 16, pp. 1603–1603, 2021.
S. E. Atalah, “Epidemiología de la obesidad en chile,”Revista Médica Clínica Las Condes, vol. 23, no. 2, pp. 117–123, 2012.
L. Zepeda and D. Deal, “Think before you eat: photographic food diaries as intervention tools to change dietary decision making and attitudes,”Int. J. of Consumer Studies, vol. 32, no. 6, pp. 692–698, 2008.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez,Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advancesin Neural Information Processing Systems, pp. 5998–6008, 2017.
M. Chen, K. Dhingra, W. Wu, L. Yang, R. Sukthankar, and J. Yang,“Pfid: Pittsburgh fast-food image dataset,” in2009 16th IEEE ICIP,pp. 289–292, IEEE, 2009.
T. Joutou and K. Yanai, “A food image recognition system with multiple kernel learning,” in 2009 16th IEEE ICIP, pp. 285–288, IEEE, 2009.
M. Bosch, F. Zhu, N. Khanna, C. J. Boushey, and E. J. Delp, “Combining global and local features for food identification in dietary assessment,”in2011 18th IEEE ICIP, pp. 1789–1792, IEEE, 2011.
V. Bettadapura, E. Thomaz, A. Parnami, G. D. Abowd, and I. Essa, “Leveraging context to support automated food recognition in restaurants,”in2015 IEEE WACV, pp. 580–587, IEEE, 2015.
S. Yang, M. Chen, D. Pomerleau, and R. Sukthankar, “Food recognition using statistics of pairwise local features,” in2010 IEEE Computer Society Conf. CVPR, pp. 2249–2256, IEEE, 2010.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,”Advances in Neural Informa-tion Processing Systems, vol. 25, pp. 1097–1105, 2012.
H. Hassannejad, G. Matrella, P. Ciampolini, I. De Munari, M. Mor-donini, and S. Cagnoni, “Food image recognition using very dee pconvolutional networks,” in Proc. 2nd Int. Workshop MADiMa, pp. 41–49, 2016.
K. Yanai and Y. Kawano, “Food image recognition using deep convolutional network with pre-training and fine-tuning,” in2015 IEEE ICMEW,pp. 1–6, IEEE, 2015.
N. Martinel, G. L. Foresti, and C. Micheloni, “Wide-slice residual networks for food recognition,” in 2018 IEEE WACV, pp. 567–576,IEEE, 2018.
E. Aguilar, M. Bolaños, and P. Radeva, “Food recognition using fusion of classifiers based on cnns,” in ICIAP, pp. 213–224, Springer, 2017.
E. Tasci, “Voting combinations-based ensemble of fine-tuned convolutional neural networks for food image recognition,”Multimedia Toolsand Applications, vol. 79, no. 41, pp. 30397–30418, 2020.
S. Zagoruyko and N. Komodakis, “Wide residual networks,”arXivpreprint arXiv:1605.07146, 2016.
S. Jiang, W. Min, L. Liu, and Z. Luo, “Multi-scale multi-view deep feature aggregation for food recognition,”IEEE Trans. Image Process.,vol. 29, pp. 265–276, 2019.
J. Qiu, F. P. W. Lo, Y. Sun, S. Wang, and B. Lo, “Mining discriminative food regions for accurate food recognition,” 2019.
W. Min, L. Liu, Z. Wang, Z. Luo, X. Wei, X. Wei, and S. Jiang, “Isiafood-500: A dataset for large-scale food recognition via stacked global-local attention network,” inProc. 28th ACM Int. Conf. on Multimedia,pp. 393–401, 2020.
Y. Wei, J. Feng, X. Liang, M.-M. Cheng, Y. Zhao, and S. Yan, “Object region mining with adversarial erasing: A simple classification to seman-tic segmentation approach,” in Proc. IEEE Conf. CVPR, pp. 1568–1576,2017.
J. Chen and C.-W. Ngo, “Deep-based ingredient recognition for cooking recipe retrieval,” in Proc. 24th ACM Int. Conf. on Multimedia, pp. 32–41,2016.
J.-j. Chen, C.-W. Ngo, and T.-S. Chua, “Cross-modal recipe retrieval with rich food attributes,” in Proc. 25th ACM Int. Conf. on Multimedia,pp. 1771–1779, 2017.
E. Aguilar, M. Bolaños, and P. Radeva, “Regularized uncertainty-based multi-task learning model for food analysis,”Journal of Visual Communication and Image Representation, vol. 60, pp. 360–370, 2019.
T. Ege and K. Yanai, “Simultaneous estimation of food categories and calories with multi-task cnn,” in 2017 15th IAPR Int. Conf. MVA,pp. 198–201, IEEE, 2017.
H. Wu, M. Merler, R. Uceda-Sosa, and J. R. Smith, “Learning to make better mistakes: Semantics-aware visual food recognition,” in Proc. 24thACM Int. Conf. on Multimedia, pp. 172–176, 2016.
R. Mao, J. He, Z. Shao, S. K. Yarlagadda, and F. Zhu, “Visual aware hierarchy based food recognition,” in ICPR Workshops (5), pp. 571–598,2020.
E. Aguilar and P. Radeva, “Uncertainty-aware integration of local and flat classifiers for food recognition,”Pattern Recognition Letters,vol. 136, pp. 237–243, 2020.
H. Zhao, K.-H. Yap, A. C. Kot, and L. Duan, “Jdnet: A joint-learning distilled network for mobile visual food recognition,”IEEE Journal of Selected Topics in Signal Processing, vol. 14, no. 4, pp. 665–675, 2020.
H. Zhao, K.-H. Yap, and A. C. Kot, “Fusion learning using semantics and graph convolutional network for visual food recognition,” in Proc.IEEE/CVF WACV, pp. 1711–1720, 2021.
A. Bera, Z. Wharton, Y. Liu, N. Bessis, and A. Behera, “Attend andguide (ag-net): A key points-driven attention-based deep network for image recognition,”IEEE Trans. Image Process., vol. 30, pp. 3691–3704, 2021.
L. Bossard, M. Guillaumin, and L. Van Gool, “Food-101–mining discriminative components with random forests,” in ECCV, pp. 446–461,Springer, 2014.
Y. Matsuda, H. Hoashi, and K. Yanai, “Recognition of multiple-food images by detecting candidate regions,” in2012 IEEE ICME, pp. 25–30, IEEE, 2012.
Y. Kawano and K. Yanai, “Automatic expansion of a food image dataset leveraging existing categories with domain adaptation,” inECCV, pp. 3–17, Springer, 2014.
S. Mezgec and B. Koroušic Seljak, “Nutrinet: a deep learning food and drink image recognition system for dietary assessment,”Nutrients, vol. 9,no. 7, p. 657, 2017.
S. Hou, Y. Feng, and Z. Wang, “Vegfru: A domain-specific dataset for fine-grained visual categorization,” in Proc. IEEE ICCV, pp. 541–549,2017.
C. Termritthikun, P. Muneesawang, and S. Kanprachar, “Nu-innet:Thai food image recognition using convolutional neural networks on smartphone,”JTEC, vol. 9, no. 2-6, pp. 63–67, 2017.
C. Güngör, F. Baltacı, A. Erdem, and E. Erdem, “Turkish cuisine: A benchmark dataset with turkish meals for food recognition,” in 201725th SIU Conf., pp. 1–4, IEEE, 2017.
S. Aslan, G. Ciocca, D. Mazzini, and R. Schettini, “Benchmarking algorithms for food localization and semantic segmentation,”Int. J. of Mach. Learn. and Cybernetics, vol. 11, no. 12, pp. 2827–2847, 2020.
T. Ege, W. Shimoda, and K. Yanai, “A new large-scale food imagesegmentation dataset and its application to food calorie estimation basedon grains of rice,” in Proc. 5th Int. Workshop MADiMa, pp. 82–87, 2019.
K. Okamoto and K. Yanai, “Uec-foodpix complete: A large-scale food image segmentation dataset.,” inICPR Workshops (5), pp. 647–659,2020.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. CVPR, pp. 770–778, 2016.
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proc. IEEE Conf. CVPR,pp. 4700–4708, 2017.
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,”arXiv preprint arXiv:2103.14030, 2021.
