BiliScreen: Smartphone-Based Scleral Jaundice Monitoring for Liver and Pancreatic Disorders

Abstract

Pancreatic cancer has one of the worst survival rates amongst all forms of cancer because its symptoms manifest later into the progression of the disease. One of those symptoms is jaundice, the yellow discoloration of the skin and sclera due to the buildup of bilirubin in the blood. Jaundice is only recognizable to the naked eye in severe stages, but a ubiquitous test using computer vision and machine learning can detect milder forms of jaundice. We propose BiliScreen, a smartphone app that captures pictures of the eye and produces an estimate of a person's bilirubin level, even at levels normally undetectable by the human eye. We test two low-cost accessories that reduce the effects of external lighting: (1) a 3D-printed box that controls the eyes' exposure to light and (2) paper glasses with colored squares for calibration. In a 70-person clinical study, we found that BiliScreen with the box achieves a Pearson correlation coefficient of 0.89 and a mean error of -0.09 ± 2.76 mg/dl in predicting a person's bilirubin level. As a screening tool, BiliScreen identifies cases of concern with a sensitivity of 89.7% and a specificity of 96.8% with the box accessory.

References

1. Alireza Abdolvahabi, Brandon W. Taylor, Rebecca L. Holden, Elizabeth V Shaw, Alex Kentsis, Carlos Rodriguez-Galindo, Shizuo Mukai, and Bryan F. Shaw. Colorimetric and longitudinal analysis of leukocoria in recreational photographs of children with retinoblastoma. PloS one, 8(10):e76677, oct 2013.
2. American Cancer Society. Cancer Facts 8 Figures 2016. Technical report, American Cancer Society, Atlanta, GA, 2016.
3. Andrew Bastawrous, Hillary K. Rono, Iain A. T. Livingstone, Helen A. Weiss, Stewart Jordan, Hannah Kuper, and Matthew J. Burton. Development and Validation of a Smartphone-Based Visual Acuity Test (Peek Acuity) for Clinical Practice and Community-Based Fieldwork. JAMA Ophthalmology, 133(8):930, aug 2015.
4. Vinod K Bhutani, Lois Johnson, and Emidio M Sivieri. Predictive ability of a predischarge hour-specific serum bilirubin for subsequent significant hyperbilirubinemia in healthy term and near-term newborns. Pediatrics, 103(1):6--14, 1999.
5. Giles Bond-Smith, Neal Banga, Toby M Hammond, and Charles J Imber. Pancreatic adenocarcinoma. BMJ, 344, 2012.
6. Yuri Y Boykov and Marie-Pierre Jolly. Interactive graph cuts for optimal boundary 8 region segmentation of objects in ND images. In Proc. ICCV ’01, volume 1, pages 105--112. IEEE, 2001.
7. Simona Crihalmeanu and Arun Ross. Multispectral scleral patterns for ocular biometric recognition. Pattern Recognition Letters, 33(14):1860--1869, 2012.
8. Abhijit Das, Umapada Pal, Miguel Angel Ferrer Ballester, and Michael Blumenstein. A new efficient and adaptive sclera recognition system. In 2014 IEEE Symposium on Computational Intelligence in Biometrics and Identity Management (CIBIM), pages 1--8. IEEE, dec 2014.
9. Abhijit Das, Umapada Pal, Miguel Angel Ferrer Ballester, and Michael Blumenstein. Sclera recognition using dense-SIFT. In International Conference on Intelligent Systems Design and Applications, ISDA, pages 74--79. IEEE, dec 2014.
10. Lilian de Greef, Mayan Goel, Min Joon Seo, Eric C Larson, James W Stout, James A Taylor, and Shwetak N Patel. Bilicam: using mobile phones to monitor newborn jaundice. In Proc. UbiComp ’14, pages 331--342, 2014.
11. Maria Syl D De La Cruz, Alisa P Young, and Mack T Ruffin. Diagnosis and management of pancreatic cancer. American family physician, 89(8):626--32, apr 2014.
12. Mario E Giardini, Iain A T Livingstone, Stewart Jordan, Nigel M Bolster, Tunde Peto, Matthew Burton, and Andrew Bastawrous. A smartphone based ophthalmoscope. Proc. EMBC ’14, 2014:2177--2180, 2014.
13. Dorothy M Greig, Bruce T Porteous, and Allan H Seheult. Exact maximum a posteriori estimation for binary images. Series of the Royal Statistical Society, 51(2):271--279, 1989.
14. G. Heath. The episclera, sclera and conjunctiva. An overview of relevant ocular anatomy. Differential Diagnosis of Ocular Disease, 9(2):36--42, 2006.
15. Gudrun J Klinker, Steven A Shafer, and Takeo Kanade. A physical approach to color image understanding. International Journal of Computer Vision, 4(1):7--38, 1990.
16. LF Kozachenko and NN Leonenko. Sample estimate of the entropy of a random vector. Problemy Peredachi Informatsii, 23(2):9--16, 1987.
17. Everett Lawson, Jason Boggess, Siddharth Khullar, Alex Olwal, Gordon Wetzstein, and Ramesh Raskar. Computational retinal imaging via binocular coupling and indirect illumination. In Proc. SIGGRAPH ’12, page 51, 2012.
18. Somsak Leartveravat. Transcutaneous bilirubin measurement in full term neonate by digital camera. Medical Journal of Srisaket Surin Buriram Hospitals, 24(1):105--118, 2009.
19. Terence S Leung, Karan Kapur, Ashley Guilliam, Jade Okell, Bee Lim, Lindsay W MacDonald, and Judith Meek. Screening neonatal jaundice based on the sclera color of the eye using digital photography. Biomedical optics express, 6(11):4529--4538, 2015.
20. Rainer Lienhart and Jochen Maydt. An extended set of Haar-like features for rapid object detection. In Proceedings. International Conference on Image Processing, volume 1, pages 0--3, 2002.
21. Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 07-12-June, pages 3431--3440, mar 2015.
22. Dan L Longo, Anthony S Fauci, Dennis L Kasper, Stephen L Hauser, J Larry Jameson, and Joseph Loscalzo. Harrison’s Principles of Internal Medicine. 18th edition, 2006.
23. Marco Marcon, Eliana Frigerio, and Stefano Tubaro. Sclera segmentation for gaze estimation and iris localization in unconstrained images. In CompIMAGE, pages 25--29, 2012.
24. Nobuyuki Otsu. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1):62--66, jan 1979.
25. Christopher G Owen, Richard SB Newsom, Alicja R Rudnicka, Sarah A Barman, and E Goeffrey Woodward. Diabetes and the tortuosity of vessels of the bulbar conjunctiva. Ophthalmology, 115(6):e27--e32, 2008.
26. Christopher G Owen, Richard SB Newsom, Alicja R Rudnicka, and Tim J Ellis. Vascular response of the bulbar conjunctiva to diabetes and elevated blood pressure. Ophthalmology, 112(10):1801--1808, 2005.
27. Vitor F Pamplona, Ankit Mohan, Manuel M Oliveira, and Ramesh Raskar. NETRA: interactive display for estimating refractive errors and focal range. ACM transactions on graphics (TOG), 29(4):77, 2010.
28. Vitor F Pamplona, Erick B Passos, Jan Zizka, Manuel M Oliveira, Everett Lawson, Esteban Clua, and Ramesh Raskar. Catra: cataract probe with a lightfield display and a snap-on eyepiece for mobile phones. In Proc. SIGGRAPH ’11, pages 7--11, 2011.
29. Danny Pascale. RGB coordinates of the Macbeth ColorChecker. Technical report, 2006.
30. Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (TOG) ’04, 23(3):309--314, 2004.
31. Mario A Ruiz, Sammy Saab, and Leland S Rickman. The clinical detection of scleral icterus: observations of multiple examiners. Military medicine, 162(8):560--563, 1997.
32. Gaurav Sharma, Wencheng Wu, and Edul N. Dalal. The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Research and Application, 30(1):21--30, feb 2005.
33. DO Sillence, Alison Senn, and DM Danks. Genetic heterogeneity in osteogenesis imperfecta. Journal of medical genetics, 16(2):101--116, 1979.
34. Audrey Vincent, Joseph Herman, Rich Schulick, Ralph H Hruban, and Michael Goggins. Pancreatic cancer. The Lancet, 378(9791):607--620, aug 2011.
35. Paul Viola and Michael J Jones. Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition (CVPR), volume 1, pages 511--518, 2001.
36. Russell Wiesner, Erick Edwards, Richard Freeman, Ann Harper, Ray Kim, Patrick Kamath, Walter Kremers, John Lake, Todd Howard, Robert M. Merion, Robert A. Wolfe, Ruud Krom, and United Network for Organ Sharing Liver Disease Severity Score Committee. Model for end-stage liver disease (MELD) and allocation of donor livers. Gastroenterology, 124(1):91--96, jan 2003.
37. Stephen Wolf. Color correction matrix for digital still and video imaging systems. pages 1--40, 2003.
38. Zhi Zhou, Eliza Yingzi Du, N. Luke Thomas, and Edward J. Delp. A New Human Identification Method: Sclera Recognition. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 42(3):571--583, may 2012.

Index Terms

• Applied computing

  • Life and medical sciences

    • Consumer health

• Human-centered computing

  • Ubiquitous and mobile computing

    • Ubiquitous and mobile devices

      • Smartphones