A schema-based selective rendering framework

ABSTRACT

Perception principles have been incorporated into rendering algorithms in order to optimize rendering computation and produce photorealistic images from a human rather than a machine point of view. In order to economize on rendering computation, selective rendering guides high level of detail to specific regions of a synthetic scene and lower quality to the remaining scene, without compromising the level of information transmitted. Scene regions that have been rendered in low and high quality can be combined to form one complete scene. Such decisions are guided by predictive attention modeling, gaze or task-based information. We propose a novel selective rendering approach which is task and gaze-independent, simulating cognitive creation of spatial hypotheses. Scene objects are rendered in varying polygon quality according to how they are associated with the context (schema) of the scene. Experimental studies in synthetic scenes have revealed that consistent objects which are expected to be found in a scene can be rendered in lower quality without affecting information uptake. Exploiting such expectations, inconsistent items which are salient require a high level of rendering detail in order for them to be perceptually acknowledged. The contribution of this paper is an innovative x3D-based selective rendering framework based on memory schemas and implemented through metadata enrichment.

References

1. Bartlett, F. C. 1932. Remembering. Cambridge University Press, Cambridge, England, p-213.
2. Brewer, W. F.&Treyens, J. C. 1981. Role of Schemata in Memory for Places. Cognitive Psychology, 13, 207--230.
3. Brooks Jr, F. P. 1999. What's Real About Virtual Reality. IEEE Computer Graphics and Applications, 16--27.
4. Cater, K., Chalmers, A., Ward G. 2003. Detail to Attention: Exploiting Visual Tasks for Selective Rendering. In Proceedings of the Eurographics Workshop on Rendering.
5. Ferwerda, J. 2001. Hi-Fi rendering. ACM Siggraph Eurographics campfire on perceptually adaptive graphics. http://isg.cs.tcd.ie/campfire/jimferwerda2.html.
6. Flannery, K. A. Walles, R. 2003. How does schema theory apply to real versus virtual memories? Cyberspychology and Behavior, 6, 2, 151--159.
7. Haber, J., Myszkowski, K., Yamauchi, H., Seidel, H. P. 2001. Perceptually guided corrective splatting, Computer Graphics Forum, 20, 3, 142--152.
8. Itti, L., Koch, C., Neibur, E. 1998. A Model of Saliency-based Visual Attention for Rapid Scene Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254--1259.
9. Land. M. F. 1999. Motion and vision: why animals move their eyes. Journal of Comparative Physiology A: Neuroethology, Sensory, Neural, and Behavioural Physiology, 185, 4, 341--352, 1999.
10. Loftus, G. R., Mackworth, N. H. 1978. Cognitive determinants of fixation location during picture viewing. Educational Psychology: Human Perception and Performance, 4, 4, 565--572, 1978.
11. Maconcie, G. W., Loschky, L. C. 1997. Human performance with a gaze linked multi-resolutional display. In Proceedings of the Advanced Displays and Interactive Displays First Annual Symposium 1997, 25--34. ARL Federated laboratory advanced displays and interactive displays consortium.
12. Mania, K.&Chalmers, A. 2001. The Effects of Levels of Immersion on Presence and Memory in Virtual Environments: A Reality Centred Approach. Cyberpsychology&Behavior, 4(2), 247--264.
13. Mania, K., Adelstein, B., Ellis, S. R., Hill, M. 2004. Perceptual Sensitivity to Head Tracking Latency in Virtual Environments with Varying Degrees of Scene Complexity. In Proceedings of the Symposium on Applied Perception in Graphics and Visualization 2004, ACM, 39--47.
14. Mania, K., Robinson, A. 2003. Simulating Spatial Assumptions. Technical Sketch. ACM SIGGRAPH 2003, San Diego, USA.
15. Mania, K., Robinson, A. 2004. The Effect of Quality of Rendering on User Lighting Impressions and Presence in Virtual Environments. In Proceedings of the ACM Siggraph International conference on Virtual Reality Continuum and its Applications in Industry 2004, Singapore. ACM, 200--205.
16. Mania, K., Troscianko, T., Hawkes, R., Chalmers, A. 2003. Fidelity Metrics for Virtual Environment Simulations based on Human Judgments of Spatial Memory Awareness States. Presence, Teleoperators and Virtual Environments, 12(3), 296--310, MIT Press.
17. Mania, K., Robinson, A., Brandt, K. 2005. The effect of memory schemata on object recognition in virtual environments, Presence, Teleoperators and Virtual Environments, 14, 5, 606--615.
18. Marmitt, G., Duchowski, A. T. 2002. Modeling visual attention in VR: measuring the accuracy of predicted scanpaths. In Proceedings of Eurographics 2002, Short Presentations, 217--226.
19. McNamara, A. 2001. Visual Perception in Realistic Image Synthesis', Computer Graphics Forum, Volume 20, #4.
20. Mourkoussis, N., Rivera, F., Troscianko, T., Dixon, T., Hawkes, R., Mania, K. Accepted for publication 2009. Quantifying Fidelity for Virtual Environment simulations employing memory schema assumptions. ACM Transactions on Applied Perception.
21. Rojan, K., Pettigrew, T. F. 1992. Memory for schema-relevant information: a meta-analytic resolution. British Journal of Social Psychology, 31, 2, 81--109.
22. Rushmeier, H., Ward, G., Piatco, C., Sanders, P., Rust, B. 1995 Comparing Real and Synthetic Images: Some ideas about metrics. In Proceedings of EGRW 1995.
23. Sundstedt, V., Stavrakis, E., Wimmer, M., Reinhard, E. 2008. A Psychophysical Study of Fixation Behavior in a Computer Game. In Proceedings of APGV 2008 - Symposium on Applied Perception in Graphics and Visualization 2008, 43--50.
24. Troscianko, T., Mourkoussis, N., Rivera, F., Mania, K., Dixon, T., Hawkes, R. 2007. Memory for objects in virtual environments, Journal of Vision, Volume 7, Number 9, 763a.
25. Watson, B. A., Walker, N., Hodges, L. F., Reddy, M. 1997. An evaluation of level of detail degradation in head-mounted display peripheries. Presence: Teleoperators and Virtual Environments, 6, 6, 630--637, 1997.
26. T. D. Wickens. 2001. Elementary Signal Detection Theory. New York: Oxford University Press.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

        • Scene understanding

      • Image and video acquisition

        • 3D imaging

  • Computer graphics

    • Animation

    • Image manipulation

    • Rendering

  • Modeling and simulation

    • Model development and analysis

      • Modeling methodologies