Annotating historical archives of images

ABSTRACT

Recent initiatives like the Million Book Project and Google Print Library Project have already archived several million books in digital format, and within a few years a significant fraction of world's books will be online. While the majority of the data will naturally be text, there will also be tens of millions of pages of images. Many of these images will defy automation annotation for the foreseeable future, but a considerable fraction of the images may be amiable to automatic annotation by algorithms that can link the historical image with a modern contemporary, with its attendant metatags. In order to perform this linking we must have a suitable distance measure which appropriately combines the relevant features of shape, color, texture and text. However the best combination of these features will vary from application to application and even from one manuscript to another. In this work we propose a simple technique to learn the distance measure by perturbing the training set in a principled way. We show the utility of our ideas on archives of manuscripts containing images from natural history and cultural artifacts.

REFERENCES

• 1: T. Adamek, N. E. O'Connor, and A. F. Smeaton. Word matching using single closed contours for indexing handwritten historical documents. IJDAR 9(2-4): 153--165 (2007).
• 2: M. Agosti, N. Ferro, and N. Orio. Annotations as a Tool for Disclosing Hidden Relationships Between Illuminated Manuscripts. AI*IA 2007: 662--673.
• 3: A. Antonacopoulos, and A. C. Downton. Special issue on the analysis of historical documents. IJDAR 9(2-4): 75--77 (2007).
• 4: X. Chen, and T.-J. Cham. Learning Feature Distance Measures for Image Correspondences. CVPR (2) 2005: 560--567.
• 5: S. F. Denton (1890). Report of the Fish and Game commission of The State of New York. Seventh Report.
• 6: C. D'Orbigny (1849). Dictionnaire Universel d'Histoire Naturelle. Renard & Martinet, Paris.
• 7: A. Ernst-Gerlach, N. Fuhr (2007) Retrieval in text collections with historic spelling using linguistic and spelling variants. JCDL 2007: 333--341
• 8: P. S. Fres (1991). The Illustrated Encyclopedia of the Butterfly World, by, 1991. Tiger Books International PLC, London or 1989, Crescent Books, Crown Publishers Inc, NY.
• 9: P. Gregory. (1958). Guide to the identification of certain American Indian projectile points. The Norman Society. Special bulletin no. 4 of the Oklahoma Anthropological Society.
• 10: M. Herwig (2007). GOOGLE'S TOTAL LIBRARY: Putting the World's Books on the Web. http://www.spiegel.de/international/.
• 11: T. Joachims. Making large-scale support vector machine learning practical. In Advances in Kernel Methods: Support Vector Machines. MIT Press, Cambridge, MA, 1998.
• 12: K. Kelly (2006). Scan This Book! N.Y. TIMES, May 14, § 6 (Magazine), at 42.
• 13: E. Keogh, L. Wei, X. Xi, S. H. Lee, and M. Vlachos. LB_Keogh Supports Exact Indexing of Shapes under Rotation Invariance with Arbitrary Representations and Distance Measures. In Proceedings of Very Large Databases (VLDB'06), 2006, pp 882--893.
• 14: S. Mahamud (2002). Discriminative distance measures for object detection. Doctoral dissertation, CMU. www-2.cs.cmu.edu/~mahamud/recognition/recognition.html.
• 15: T. Moffett [Muffet] (1658). The Theatre of Insects; or, Lesser living Creatures, as Bees, Flies, Caterpillars, Spiders, Worms, etc., a most Elaborate Work in vol. 2 of Edward Topsell, The History of Four-footed Beasts and Serpents: Describing at Large Their True and Lively Figure, their several Names, Conditions, Kinds, Virtues (both Natural and Medicinal) Countries of their Breed, their Love and Hatred to Mankind. Collected out of the writings of Conradus Gesner and other authors, by Edward Topsell. Whereunto is now added, The Theater of Insects... by T. Muff et, 2 vols. London: printed by E. Cotes, for G. Sawbridge, T. Williams, and T. Johnson, 1658.
• 16: G. P. Nguyen, M. Worring, and A. W. M. Smeulders. Interactive search by direct manipulation of dissimilarity space. IEEE Transactions on Multimedia. VOL. 9, NO. 7, Nov 2007.
• 17: C. B. Richard Ng, G. Lu, and D. Zhang. Performance Study of Gabor Filters and Rotation Invariant Gabor Filters. MMM 2005: 158--162.
• 18: A. Seba (1734). Locupletissimi rerum naturalium thesauri accurata descriptio Naaukeurige beschryving van het schatryke kabinet der voornaamste seldzaamheden der natuur. Amsterdam, 1734--1765. 4 vols. 2º. - 394 B 26--29, vol. 3, plate XXXV.
• 19: W. Smith. (1853) British Diatomaceae. Volume 1. John Van Voorst, London: xxxiv+89pp.
• 20: S. Squire. (1998) Learning a Similarity-Based Distance Measure for Image Database Organization from Human Partitionings of an Image Set. IEEE Workshop on Applications of Computer Vision (WACV'98), pp.88--93, 1998.
• 21: Varde, A., Rundensteiner, E., Javidi, G., Sheybani, E. and Liang J. (2007). Learning the Relative Importance of Features in Image Data. In Proceedings of IEEE ICDE's DBRank-07, Istanbul, Turkey, April 2007, pp. 237--244
• 22: R. C. Veltkamp, and L. J. Latecki. Properties and Performance of Shape Similarity Measures. In Proceedings of IFCS 2006 Conference: Data Science and Classification. July, 2006.
• 23: X. Wang, L. Ye, E. Keogh, and C. Shelton (2008). www.cs.ucr.edu/~xwang/historical_shapes/index.html.
• 24: www.entomo.pl/lepidoptera/galeria_motyli/motyle_nocne.htm Visited on 26-Oct-07.
• 25: www.temple-of-flora.com/natural_history.htm Visited on 28-Oct-07.
• 26: www.texasarrowheads.com/ Visited on 15-Jan-08.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms

Keywords:
historical digital libraries, historical manuscripts, image annotation, information extraction