EXPRESS: a data EXtraction, Processing, and Restructuring System

ABSTRACT

EXPRESS is an experimental prototype data translation system which can access a wide variety of data and restructure it for new uses. The system is driven by two very high level nonprocedural languages: DEFINE for data description and CONVERT for data restructuring. Program generation and cooperating process techniques are used to achieve efficient operation. This paper describes the design and implementation of EXPRESS. DEFINE and CONVERT are summarized and the implementation architecture presented. The DEFINE description is compiled into a customized PL/1 program for accessing source data. The restructuring specified in CONVERT is compiled into a set of customized PL/1 procedures to derive multiple target files from multiple input files. Job steps and job control statements are generated automatically. During execution, the generated procedures run under control of a process supervisor, which coordinates buffer management and handles file allocation, deallocation, and all input/output requests. The architecture of EXPRESS allows efficiency in execution by avoiding unnecessary secondary storage references while at the same time allowing the individual procedures to be independent of each other. Its modular structure permits the system to be extended or transferred to another environment easily.

REFERENCES

• 1: David E. Bakkom , James A. Behymer, Implementation of a prototype generalized file translator, Proceedings of the 1975 ACM SIGMOD international conference on Management of data, May 14-16, 1975, San Jose, California [doi>10.1145/500080.500095]
• 2: BIRSS, E.W., AND Far, J.P. Generalized software for translating data. Proc. AFIPS 1976 NCC, AFIPS Press, Montvale, N.J., pp. 889-899.
• 3: DEPPE, M., LEWIS, K., AND SWARTWOUT, D. Operational software for restructuring network data bases. Working Paper DT 3.2, Data Translation Proj., U. of Michigan, Ann Arbor, Mich., 1976.
• 4: James P. Fry , Randall L. Frank , Ernest A. Hershey, III, A developmental model for data translation, Proceedings of 1972 ACM-SIGFIDET workshop on Data description, access and control, p.77-105, January 1972 [doi>10.1145/800295.811486]
• 5: James P. Fry , Diane P. Smith , Robert W. Taylor, An approach to Stored Data Definition and Translation, Proceedings of 1972 ACM-SIGFIDET workshop on Data description, access and control, p.13-55, January 1972 [doi>10.1145/800295.811483]
• 6: FRY, J.P., SMITH, D.C.P., TAYLOR, R.W., FRANK, R.L., LUM, V.Y., BEHYMER, J.A., AND SHNEID~.R~AN, B. Stored-data description and data translation: A model and language. Inform. Syst. 2, 3 (1977), 95-160.
• 7: Barron C. Housel , Vincent Y. Lum , Nan Shu, Architecture to an interactive migration system (AIMS), Proceedings of the 1974 ACM SIGFIDET (now SIGMOD) workshop on Data description, access and control, p.157-169, May 01-03, 1974, Ann Arbor, Michigan [doi>10.1145/800296.811510]
• 8: Barron C. Housel , Nan C. Shu, A high-level data manipulation language for hierarchical data structures, Proceedings of the 1976 conference on Data : Abstraction, definition and structure, p.155-169, March 22-24, 1976, Salt Lake City, Utah, United States
• 9: HOUSEL, B.C., SMITH, D.P., SHU, N.C., AND LuM, V.Y. DEFINE--A nonprocedural data description language for de/ining information easily. Proc. ACM Pacific 75, San Francisco, Calif., April 1975, pp. 62-70.
• 10: IBM CORP. Information Management System, General Information Manual. IBM: Pub. No. GH20-1260, IBM Corp., White Plains, N.Y., 1975.
• 11: LooMIs, M.E. Resource-constrained scheduling of tasks with precedence relationships. Res. Rep. No. RJ 1746, IBM Res. Lab., San Jose, Calif., 1976.
• 12: Lug, V.Y., SHU, N.C., AND HOVSEL, B.C. A general methodology for data conversion and restructuring. IBM J. Res. and Develop. 20, 5 (1976), 483--497.
• 13: Alan G. Merten , James P. Fry, A data description language approach to file translation, Proceedings of the 1974 ACM SIGFIDET (now SIGMOD) workshop on Data description, access and control, p.191-205, May 01-03, 1974, Ann Arbor, Michigan [doi>10.1145/800296.811512]
• 14: Shamkant B. Navathe , James P. Fry, Restructuring for large databases: three levels of abstraction, ACM Transactions on Database Systems (TODS), v.1 n.2, p.138-158, June 1976 [doi>10.1145/320455.320461]
• 15: RAMIR~.Z, J.A. Automalic Generation of Data Conversion Programs Using a Dala Description Language (DDL), Vols. 1, H. U. of Pennsylvania, Philadelphia, Pa., May 1973.
• 16: J. A. Ramirez , N. A. Rin , N. S. Prywes, “Automatic generation of data conversion programs using a data description language”, Proceedings of the 1974 ACM SIGFIDET (now SIGMOD) workshop on Data description, access and control, p.207-225, May 01-03, 1974, Ann Arbor, Michigan [doi>10.1145/800296.811513]
• 17: SCHNEIDER, G.M., AND DESAUTELS, E.J. Design of a file translation language for networks. Inform. Syst. 1, 1 (Jan. 1975), 23-31.
• 18: SI~S~.ID~RMAN, B., XND SHAPIRO, S.C. Towards a theory of encoded data structures and data translation. Int. J. Comptr. Inform. Sci. 5, 1 (1976), 33-43.
• 19: Arie Shoshani, A logical-level approach to data base conversion, Proceedings of the 1975 ACM SIGMOD international conference on Management of data, May 14-16, 1975, San Jose, California [doi>10.1145/500080.500097]
• 20: SHOSHANI, A., AND BRANDON, K. On the implementation of a logical data base converter. Proc. Int. Conf. on Very Large Data Bases, Framingham, Mass., Sept. 1975, pp. 529-531.
• 21: Nan C. Shu , Barron C. Housel , Vincent Y. Lum, CONVERT: a high level translation definition language for data conversion, Communications of the ACM, v.18 n.10, p.557-567, Oct. 1975 [doi>10.1145/361020.361023]
• 22: Edgar H. Sibley , Robert W. Taylor, A data definition and mapping language, Communications of the ACM, v.16 n.12, p.750-759, Dec. 1973 [doi>10.1145/362552.362555]
• 23: SmTn, D.P. An approach to data description and conversion. Ph.D. Diss., U. of Pennsylvania, Philadelphia, Pa., 1971.
• 24: Diane C. P. Smith, A method for data translation using the Stored-Data Definition and Translation Task Group languages, Proceedings of 1972 ACM-SIGFIDET workshop on Data description, access and control, p.107-124, January 1972 [doi>10.1145/800295.811487]
• 25: Robert Walker Taylor, Generalized data base management system data structures and their mappingto physical storage, 1971
• 26: WINTERS, E.W., AND DICKEY, A.F. A business application of data translation. Proc. 1976 ACM SIGMOD Conf., Washington, D.C., June 1976, pp. 189-196.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.5 Heterogeneous Databases
          Subjects: Data translation**

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.2 Information Storage
          Subjects: File organization

General Terms:
Languages, Theory

Keywords:
data conversion, data description languages, data manipulation languages, data restructuring, data translation, file conversion, program generation, very high level languages