Image-based food volume estimation

ABSTRACT

In this paper, we propose an extension to our previous work on food portion size estimation using a single image and a multi-view volume estimation method. The single-view technique estimates food volume by using prior information (segmentation and food labels) generated from food identification methods we described earlier. For multi-view volume estimation, we use ``Shape from Silhouettes'' to estimate the food portion size. The experimental results of our volume estimation methods demonstrate our results with respect to accuracy and reliability.

References

1. M. Bosch, T. Schap, F. Zhu, N. Khanna, C. Boushey, and E. Delp. Integrated database system for mobile dietary assessment and analysis. In Proceedings of the 1st IEEE International Conference Workshop on Multimedia Services and Technologies for E-health in conjunction with the International Conference on Multimedia and Expo, pages 1--6, Barcelona, Spain, July 2011.
2. M. Bosch, F. Zhu, N. Khanna, C. Boushey, and E. Delp. Combining global and local features for food identification and dietary assessment. In Proceedings of the International Conference on Image Processing, pages 1789--1792, Brussels, Belgium, September 2011.
3. C. J. Boushey, D. A. Kerr, J. Wright, K. D. Lutes, D. S. Ebert, and E. J. Delp. Use of technology in children's dietary assessment. European Journal of Clinical Nutrition, 63:50--57, February 2009.
4. J. Chae, I. Woo, S. Kim, R. Maciejewski, F. Zhu, E. Delp, C. Boushey, and D. Ebert. Volume estimation using food specific shape templates in mobile image-based dietary assessment. In Proceedings of the IS&T/SPIE Conference on Computational Imaging IX, volume 7873, pages 78730K-1-8, San Francisco, CA, February 2011.
5. H. Chen, W. Jia, Z. Li, Y. Sun, and M. Sun. 3D/2D model-to-image registration for quantitative dietary assessment. In Proceedings of 38th Annual Northeast Bioengineering Conference, pages 95--96, March 2012.
6. B. Curless and M. Levoy. A volumetric method for building complex models from range images. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pages 303--312. ACM, August 1996.
7. B. L. Daugherty, T. E. Schap, R. Ettienne-Gittens, F. Zhu, M. Bosch, E. J. Delp, D. S. Ebert, D. A. Kerr, and C. J. Boushey. Novel technologies for assessing dietary intake: Evaluating the usability of a mobile telephone food record among adults and adolescents. Journal of Medical Internet Research, 14(2):156--167, April 2012.
8. R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, second edition, 2004.
9. Y. He, N. Khanna, C. Boushey, and E. Delp. Snakes assisted food image segmentation. In Proceedings of IEEE International Workshop on Multimedia Signal Processing, pages 181--185, Banff, Canada, September 2012.
10. Y. He, C. Xu, N. Khanna, C. Boushey, and E. Delp. Food image analysis: Segmentation, identification and weight estimation. In Proceedings of the IEEE International Conference on Multimedia and Expo, San Jose, CA, July 2013.
11. F. S. Hill Jr and S. M. Kelley. Computer Graphics Using OpenGL. Pearson Press, third edition, 2006.
12. W. Jia, Y. Yue, J. Fernstrom, Z. Zhang, Y. Yang, M. Sun, et al. 3D localization of circular feature in 2D image and application to food volume estimation. In Proceeding of 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 4545--4548, March 2012.
13. S. Kelkar, S. Stella, C. Boushey, and M. Okos. Developing novel 3D measurement techniques and prediction method for food density determination. Procedia Food Science, 1:483--491, 2011.
14. F. Kong and J. Tan. Dietcam: Regular shape food recognition with a camera phone. In Proceedings of the International Conference on Body Sensor Networks, pages 127--132, May 2011.
15. F. Kong and J. Tan. Dietcam: Automatic dietary assessment with mobile camera phones. Pervasive and Mobile Computing, 8(1):147--163, 2012.
16. K. N. Kutulakos and S. M. Seitz. A theory of shape by space carving. International Journal of Computer Vision, 38(3):199--218, 2000.
17. C. D. Lee, J. Chae, T. E. Schap, D. A. Kerr, E. J. Delp, D. S. Ebert, and C. J. Boushey. Comparison of known food weights with image-based portion-size automated estimation and adolescents' self-reported portion size. Journal of diabetes science and technology, 6(2):428, 2012.
18. T. R. E. Schap, B. L. Six, E. J. Delp, D. S. Ebert, D. A. Kerr, and C. J. Boushey. Adolescents in the united states can identify familiar foods at the time of consumption and when prompted with an image 14 h postprandial, but poorly estimate portions. Public Health Nutrition, 1(1):1--8.
19. J. Shang, M. Duong, E. Pepin, X. Zhang, K. Sandara-Rajan, A. Mamishev, and A. Kristal. A mobile structured light system for food volume estimation. In Proceeding of IEEE International Conference on Computer Vision Workshops, pages 100--101, November 2011.
20. B. M. Silva, I. M. Lopes, J. J. Rodrigues, and P. Ray. Sapofitness: a mobile health application for dietary evaluation. In Proceeding of 13th IEEE International Conference on e-Health Networking Applications and Services, pages 375--380, June 2011.
21. B. L. Six, T. E. Schap, F. Zhu, A. Mariappan, M. Bosch, E. J. Delp, D. S. Ebert, D. A. Kerr, and C. J. Boushey. Evidence-based development of a mobile telephone food record. Journal of the American Dietetic Association, 110(1):74--79, January 2010.
22. M. Sun, J. D. Fernstrom, W. Jia, S. A. Hackworth, N. Yao, Y. Li, C. Li, M. H. Fernstrom, and R. J. Sclabassi. A wearable electronic system for objective dietary assessment. Journal American Dietetic Association, 110(1):45--47, January 2010.
23. I. Woo, K. Ostmo, S. Kim, D. S. Ebert, E. J. Delp, and C. J. Boushey. Automatic portion estimation and visual refinement in mobile dietary assessment. In Proceedings of the IS&T/SPIE Conference on Computational Imaging VIII, pages 75330O--1--10, San Jose, CA, January 2010.
24. C. Xu, Y. He, N. Khanna, C. Boushey, and E. Delp. Model-based food volume estimation using 3D pose. In Proceedings of the IEEE International Conference on Image Processing, Melbourne, Australia, September 2013.
25. C. Xu, F. Zhu, N. Khanna, C. Boushey, and E. Delp. Image enhancement and quality measures for dietary assessment using mobile devices. In Proceedings of the IS&T/SPIE Conference on Computational Imaging X, pages 82960Q-1-10, San Francisco, USA, February 2012.
26. F. Zhu, M. Bosch, N. Khanna, C. Boushey, and E. Delp. Multilevel segmentation for food classification in dietary assessment. In Proceedings of the 7th International Symposium on Image and Signal Processing and Analysis, pages 337--342, Dubrovnik, Croatia, September 2011.
27. F. Zhu, M. Bosch, I. Woo, S. Kim, C. Boushey, D. Ebert, and E. Delp. The use of mobile devices in aiding dietary assessment and evaluation. IEEE Journal of Selected Topics in Signal Processing, 4(4):756 --766, August 2010.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Shape inference

  • Computer graphics

    • Shape modeling