Nicola Ferro
Gianmaria Silvello
Erik Buelinckx
ABSTRACT
We develop an evaluation framework for the validation of conformance checkers for the long-term preservation. The framework assesses the correctness, usability, and usefulness of the tools for three media types: PDF/A (text), TIFF (image), and Matroska (audio/video). Finally, we report the results of the validation of these conformance checkers using the proposed framework. In general, the presented framework is a high-level tool that can be quite easily employed in other preservation-related tasks.
- O. Alonso. Implementing crowdsourcing-based relevance experimentation: an industrial perspective. Information Retrieval, 16(2):101--120, April 2013. Google Scholar
Digital Library
- C. Becker and K. Duretec. Free Benchmark Corpora for Preservation Experiments: Using Model-Driven Engineering to Generate Data Sets. In Proc. 13th ACM/IEEECS Joint Conference on Digital Libraries (JCDL 2013), pages 349--358. ACM Press, New York, USA, 2013. Google ScholarDigital Library
- C. Becker, K. Duretec, and A. Rauber. The Challenge of Test Data Quality in Data Processing. ACM Journal of Data and Information Quality (JDIQ), 8(2), 2016. Google ScholarDigital Library
- C. Becker and A. Rauber. Decision Criteria in Digital Preservation: What to Measure and How. Journal of the American Society for Information Science and Technology (JASIST), 62(6):1009--1028, 2011. Google ScholarDigital Library
- D. Calvanese, D. De Nart, and C. Tasso, editors. Digital Libraries on the Move -- Proc. 11th Italian Research Conference on Digital Libraries (IRCDL 2015). Communications in Computer and Information Science (CCIS) 612, Springer, Heidelberg, Germany, 2016.Google Scholar
- L. Cappellato, N. Ferro, A. Fresa, M. Geber, B. Justrel, B. Lemmen, C. Prandoni, and G. Silvello. The PREFORMA Project: Federating Memory Institutions for Better Compliance of Preservation Formats. In Calvanese et al. {5}, pages 86--91.Google Scholar
- J.-P. Chanod, M. Dobreva, A. Rauber, S. Ross, and V. Casarosa. Issues in Digital Preservation: Towards a New Research Agenda. In J.-P. Chanod, M. Dobreva, A. Rauber, and S. Ross, editors, Report from Dagstuhl Seminar 10291: Automation in Digital Preservation, Dagstuhl Reports, pages 1--14. Schloss Dagstuhl--LeibnizZentrum für Informatik, Germany, 2010.Google Scholar
- C. W. Cleverdon. The Cranfield Tests on Index Languages Devices. In K. Spärck Jones and P. Willett, editors, Readings in Information Retrieval, pages 47--60. Morgan Kaufmann Publisher, Inc., San Francisco, CA, USA, 1997. Google ScholarDigital Library
- K. Duretec, A. Kulmukhametov, A. Rauber, and C. Becker. Benchmarks for Digital Preservation Tools. In Proc. 11th International Conference on Preservation of Digital Objects (iPRES 2015), 2015.Google Scholar
- K. Duretec, A. Rauber, and C. Becker. A Text Extraction Software Benchmark Based on a Synthesized Dataset. In 2017 ACM/IEEE Joint Conference on Digital Libraries, JCDL 2017, pages 109--118. IEEE Computer Society, 2017. Google ScholarDigital Library
- N. Ferro. Quality and Interoperability: The Quest for the Optimal Balance. In I. Iglezakis, T.-E. Synodinou, and S. Kapidakis, editors, E-Publishing and Digital Libraries: Legal and Organizational Issues, pages 48--68. IGI Global, USA, 2010.Google Scholar
- N. Ferro. Proposal for an Evaluation Framework for Compliance Checkers for Long-term Digital Preservation. In Digital Libraries and Multimedia Archives -- Proc. 12th Italian Research Conference on Digital Libraries (IRCDL 2016), pages 125--136. Communications in Computer and Information Science (CCIS) 701, Springer, Heidelberg, Germany, 2016.Google Scholar
- N. Ferro. Reproducibility Challenges in Information Retrieval Evaluation. ACM Journal of Data and Information Quality (JDIQ), 8(2):8:1--8:4, January 2017. Google ScholarDigital Library
- N. Ferro, E. Buelinckx, B. Doubrov, K. Jadeglans, B. Lemmens, J. Martinez, V. Muñoz, C. Prandoni, D. Rice, S. Rohde-Enslin, X. Tarrés, E. Verbruggen, B. Yousefi, and C. Wilson. Deliverable D8.1R2 -- Competitive Evaluation Strategy. PREFORMA PCP Project, EU 7FP, Contract N. 619568, October 2016.Google Scholar
- N. Ferro and G. Silvello. Towards a Semantic Web Enabled Representation of DL Foundational Models: The Quality Domain Example. In Calvanese et al. {5}, pages 24--35.Google Scholar
- N. Ferro, G. Silvello, E. Buelinckx, B. Doubrov, M. Geber, K. Jadeglans, J. Martinez, V. Muñoz, D. Rice, S. Rohde-Enslin, X. Tarrés, E. Verbruggen, B. Yousefi, and C. Wilson. Deliverable D8.6 -- Testing Report. PREFORMA PCP Project, EU 7FP, Contract N. 619568, October 2017.Google Scholar
- N. Fuhr, G. Tsakonas, T. Aalberg, M. Agosti, P. Hansen, S. Kapidakis, C.-P. Klas, L. Kovács, M. Landoni, A. Micsik, C. Papatheodorou, C. Peters, and I. Sølvberg. Evaluation of Digital Libraries. International Journal on Digital Libraries, 8(1):21-- 38, 2007. Google ScholarDigital Library
- IEC 60958. Digital audio interface - Part 1: General. Standard IEC 60958--1 Ed. 3.1 b:2014, 2014.Google Scholar
- P. Innocenti, S. Ross, E. Maceviciute, T. Wilson, J. Ludwig, and W. Pempe. Assessing Digital Preservation Frameworks: The Approach of the SHAMAN Project. In N. Spyratos, E. Kapetanios, and A. Traina, editors, Proc. ACM International Conference on Management of Emergent Digital EcoSystems (MEDES 2009), pages 412--416. ACM Press, New York, USA, 2009. Google ScholarDigital Library
- ISO 12234--2. Electronic still-picture imaging -- Removable memory -- Part 2: TIFF/EP image data format. Recommendation ISO 12234--2:2001, 2001.Google Scholar
- ISO 12639. Graphic technology -- Prepress digital data exchange -- Tag image file format for image technology (TIFF/IT). Recommendation ISO 12639:2004, 2004.Google Scholar
- ISO 14721. Space data and information transfer systems -- Open archival information system (OAIS) -- Reference model. Recom. ISO 14721:2012, 2012.Google Scholar
- ISO 19005--1. Document management -- Electronic document file format for long-term preservation -- Part 1: Use of PDF 1.4 (PDF/A-1). Recommendation ISO 19005--1:2005, 2005.Google Scholar
- ISO 19005--2. Document management -- Electronic document file format for long-term preservation -- Part 2: Use of ISO 32000--1 (PDF/A-2). Recommendation ISO 19005--2:2011, 2011.Google Scholar
- ISO 19005--3. Document management -- Electronic document file format for long-term preservation -- Part 3: Use of ISO 32000--1 with support for embedded files (PDF/A-3). Recommendation ISO 19005--3:2012, 2012.Google Scholar
- ISO/IEC 15444. Information technology -- JPEG 2000 image coding system: Core coding system. Recommendation ISO/IEC 15444--1:2004, 2004.Google Scholar
- S. T. Kowalczyk. Before the Repository: Defining the Preservation Threats to Research Data in the Lab. In P. Logasa Bogen II, S. Allard, H. Mercer, and M. Beck, editors, Proc. 15th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL 2015), pages 215--222. ACM Press, New York, USA, 2015. Google ScholarDigital Library
- S. Ross. Digital Preservation, Archival Science and Methodological Foundations for Digital Libraries. New Review of Information Networking, 17(1):43--68, 2012. Google ScholarDigital Library
- F. Sebastiani. Machine Learning in Automated Text Categorization. ACM Computing Surveys (CSUR), 34(1):1--47, March 2002. Google ScholarDigital Library
- G. Silvello. Theory and practice of data citation. JASIST, 69(1):6--20, 2018. Google ScholarDigital Library
- I. Soboroff, C. Nicholas, and P. Cahan. Ranking Retrieval Systems without Relevance Judgments. In D. H. Kraft, W. B. Croft, D. J. Harper, and J. Zobel, editors, Proc. 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001), pages 66--73. ACM Press, New York, USA, 2001. Google ScholarDigital Library
- M. Sokolova and G. Lapalme. A Systematic Analysis of Performance Measures for Classification Tasks. Information Processing &Management, 45(4):427--437, July 2009. Google ScholarDigital Library
- The Consultative Committee for Space Data Systems (CCSDS). Reference Model for an Open Archival Information System (OAIS). Magenta Book, Issue 2. Recommended Practice CCSDS 650.0-M-2, http://public.ccsds.org/publications/archive/ 650x0m2.pdf, June 2012.Google Scholar
- E. M. Voorhees. Variations in relevance judgments and the measurement of retrieval effectiveness. Information Processing &Management, 36(5):697--716, September 2000. Google ScholarDigital Library
Index Terms
- Evaluation of Conformance Checkers for Long-Term Preservation of Multimedia Documents
Recommendations
Using code analysis tools for architectural conformance checking
SHARK '11: Proceedings of the 6th International Workshop on SHAring and Reusing Architectural KnowledgeArchitectural conformance checking verifies whether a system conforms to its intended architecture, which is essential to safeguard the quality attributes of the system. Due to the size of many systems, performing conformance checking by means of manual ...
Data-aware conformance checking with SMT
AbstractConformance checking is a key process mining task to confront the normative behavior imposed by a process model with the actual behavior recorded in a log. While this problem has been extensively studied for pure control-flow processes,...
Highlights- First SMT-based approach to data-aware conformance checking via alignments.
- ...
Conformance checking based on multi-perspective declarative process models
We introduce a semantics for Multi Perspective Declare (MP-Declare).We introduce an abstract syntax for MP-Declare.We provide a set of algorithms for conformance checking based on MP-DeclareThe approach has been implemented in the process mining tool ...
Comments