Data-driven Fingertip Appearance for Interactive Hand Simulation

ABSTRACT

Contact on a finger pad results in deformation that redistributes blood within the fingertip tissue in a manner correlated to the pressure. We build a data-driven model that relates contact information to the visible changes of the finger nail and surrounding tissue on the back of the finger tip. Our data analysis and model construction makes use of the space of hemoglobin concentrations, as opposed to an RGB color space, which permits the model to be transferred across different fingers and different people. We use principal component analysis to build a compact model which maps well to graphics hardware with an efficient fragment program implementation. We provide a validation of our model, and a demonstration of a grasping controller running in a physically based simulation, where grip strength is visible in both hand posture and the appearance of color changes at the fingertips.

References

1. Aladdin, R., and Kry, P. 2012. Static pose reconstruction with an instrumented bouldering wall. In Proceedings of the 18th ACM symposium on Virtual reality software and technology, ACM, VRST '12, 177--184.
2. Bickel, B., Bächer, M., Otaduy, M. A., Matusik, W., Pfister, H., and Gross, M. 2009. Capture and modeling of non-linear heterogeneous soft tissue. ACM Trans. Graph. 28, 3 (July), 89:1--89:9.
3. Bonneel, N., van de Panne, M., Paris, S., and Heidrich, W. 2011. Displacement interpolation using lagrangian mass transport. In Proceedings of the 2011 SIGGRAPH Asia Conference, SA '11, 158:1--158:12.
4. Borshukov, G., Montgomery, J., and Hable, J. 2007. GPU Gems 3. Addison-Wesley Professional, ch. 15. Playable Universal Capture.
5. Boukhalfi, T. 2012. Automatisation des expressions faciales liÃl'es Ãă lâĂŹactivitÃl' physique. Master's thesis, ÃL'cole de technologie supÃl'rieure.
6. Donner, C., Weyrich, T., d'Eon, E., Ramamoorthi, R., and Rusinkiewicz, S. 2008. A layered, heterogeneous reflectance model for acquiring and rendering human skin. ACM Trans. Graph. 27, 5 (Dec.), 140:1--140:12.
7. Dutreve, L., Meyer, A., and Bouakaz, S. 2011. Easy acquisition and real-time animation of facial wrinkles. Comp. Anim. Virtual Worlds 22, 2-3, 169--176.
8. Huang, H., Zhao, L., Yin, K., Qi, Y., Yu, Y., and Tong, X. 2011. Controllable hand deformation from sparse examples with rich details. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA '11, 73--82.
9. Jakovels, D., Spigulis, J., and Rogule, L. 2011. RGB mapping of hemoglobin distribution in skin. In Clinical and Biomedical Spectroscopy and Imaging II, Optical Society of America, 80872B.
10. Jensen, H. W., Marschner, S. R., Levoy, M., and Hanrahan, P. 2001. A practical model for subsurface light transport. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, ACM, New York, NY, USA, SIGGRAPH '01, 511--518.
11. Jimenez, J., Scully, T., Barbosa, N., Donner, C., Alvarez, X., Vieira, T., Matts, P., Orvalho, V., Gutierrez, D., and Weyrich, T. 2010. A practical appearance model for dynamic facial color. In ACM SIGGRAPH Asia 2010 papers, SIGGRAPH ASIA '10, 141:1--141:10.
12. Kider, Jr., J. T., Pollock, K., and Safonova, A. 2011. A data-driven appearance model for human fatigue. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA '11, 119--128.
13. Kry, P. G., and Pai, D. K. 2006. Interaction capture and synthesis. ACM Trans. Graph. 25, 3 (July), 872--880.
14. Kry, P. G., James, D. L., and Pai, D. K. 2002. Eigenskin: real time large deformation character skinning in hardware. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, SCA '02, 153--159.
15. Lewis, J. P., Cordner, M., and Fong, N. 2000. Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, SIGGRAPH '00, 165--172.
16. Liu, C. K. 2009. Dextrous manipulation from a grasping pose. In ACM SIGGRAPH 2009 papers, SIGGRAPH '09, 59:1--59:6.
17. Magnenat-Thalmann, N., Laperri'ere, R., and Thalmann, D. 1988. Joint-dependent local deformations for hand animation and object grasping. In Graphics Interface '88, 26--33.
18. Mascaro, S., and Asada, H. 2004. Measurement of finger posture and three-axis fingertip touch force using fingernail sensors. Robotics and Automation, IEEE Transactions on 20, 1, 26--35.
19. Murray, R. M., Sastry, S. S., and Li, Z. 1994. A Mathematical Introduction to Robotic Manipulation. CRC Press.
20. Nishidate, I., Maeda, T., Niizeki, K., and Aizu, Y. 2013. Estimation of melanin and hemoglobin using spectral reflectance images reconstructed from a digital rgb image by the wiener estimation method. Sensors 13, 6, 7902--7915.
21. Oat, C. 2007. Animated wrinkle maps. In ACM SIGGRAPH 2007 courses, SIGGRAPH '07, 33--37.
22. Pai, D. K., Doel, K. v. d., James, D. L., Lang, J., Lloyd, J. E., Richmond, J. L., and Yau, S. H. 2001. Scanning physical interaction behavior of 3d objects. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, SIGGRAPH '01, 87--96.
23. Pollard, N. S., and Zordan, V. B. 2005. Physically based grasping control from example. In SCA '05: Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, 311--318.
24. Sloan, P.-P. J., Rose, III, C. F., and Cohen, M. F. 2001. Shape by example. In Proceedings of the 2001 symposium on Interactive 3D graphics, I3D '01, 135--143.
25. Sueda, S., Kaufman, A., and Pai, D. K. 2008. Musculotendon simulation for hand animation. ACM Transactions on Graphics 27, 3 (Aug), 83:1--83:8.
26. Tsumura, N., Haneishi, H., and Miyake, Y. 1999. Independent component analysis of skin color image. Journal of Optical Society of America A 16, 9, 2169--2176.
27. Tsumura, N., Ojima, N., Sato, K., Shiraishi, M., Shimizu, H., Nabeshima, H., Akazaki, S., Hori, K., and Miyake, Y. 2003. Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. In ACM SIGGRAPH 2003 Papers, SIGGRAPH '03, 770--779.
28. Yin, K., and Pai, D. K. 2003. Footsee: an interactive animation system. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, SCA '03, 329--338.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Image and video acquisition

        • 3D imaging

  • Computer graphics

    • Animation