research-article

A study in rule-specific issue categorization for e-rulemaking

Authors:

Stephen PurpuraAuthors Info & Claims

dg.o '08: Proceedings of the 2008 international conference on Digital government research

Pages 244 - 253

Published: 18 May 2008 Publication History

Get Access

Abstract

We address the e-rulemaking problem of categorizing public comments according to the issues that they address. In contrast to previous text categorization research in e-rulemaking [5, 6], and in an attempt to more closely duplicate the comment analysis process in federal agencies, we employ a set of rule-specific categories, each of which corresponds to a significant issue raised in the comments. We describe the creation of a corpus to support this text categorization task and report interannotator agreement results for a group of six annotators. We outline those features of the task and of the e-rulemaking context that engender both a non-traditional text categorization corpus and a correspondingly difficult machine learning problem. Finally, we investigate the application of standard and hierarchical text categorization techniques to the e-rulemaking data sets and find that automatic categorization methods show promise as a means of reducing the manual labor required to analyze large comment sets: the automatic annotation methods approach the performance of human annotators for both flat and hierarchical issue categorization.

References

[1]

S. Dumais and H. Chen. Hierarchical classification of web content. In SIGIR '00: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 256--263, New York, NY, USA, 2000. ACM.

Digital Library

Google Scholar

[2]

V. Hatzivassiloglou, J. Klavans, and E. Eskin. Detecting Text Similarity over Short Passages: Exploring Linguistic Feature Combinations via Machine Learning. In Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora (EMNLP/VLC-99), pages 203--212, University of Maryland, College Park, MD, 1999. Association for Computational Linguistics.

Google Scholar

[3]

T. Joachims. Learning to Classify Text using Support Vector Machines. Kluwer, 2002.

Crossref

Google Scholar

[4]

K. Krippendorff. Content analysis: An introduction to its methodology (2nd Ed.). Sage Publications, 2004.

Google Scholar

[5]

N. Kwon and E. Hovy. Information acquisition using multiple classifications. In Proceedings of the Fourth International Conference on Knowledge Capture (K-CAP 2007), 2007.

Digital Library

Google Scholar

[6]

N. Kwon, E. Hovy, and S. Shulman. Multidimensional text analysis for erulemaking. In Proceedings of the 7th Annual International Conference on Digital Government Research, 2006.

Digital Library

Google Scholar

[7]

M. Sahami and T. D. Heilman. A web-based kernel function for measuring the similarity of short text snippets. In WWW '06: Proceedings of the 15th international conference on World Wide Web, pages 377--386, New York, NY, USA, 2006. ACM.

Digital Library

Google Scholar

[8]

F. Sebastiani. Machine learning in automated text categorization. ACM Computing Surveys, 34(1):1--47, 2002.

Digital Library

Google Scholar

[9]

D. Silver, G. Bakir, K. Bennett, R. Caruana, M. Pontil, S. Russell, and P. Tadepalli. Inductive Transfer : 10 Years Later. NIPS 2005 Workshop, 2005.

Google Scholar

[10]

P. Strauss, T. Rakoff, and C. Farina. Administrative Law. 10th edition, 2003.

Google Scholar

[11]

V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, 1995.

Crossref

Google Scholar

[12]

H. Yang and J. Callan. Near-duplicate detection for erulemaking. In Proceedings of the Fifth National Conference on Digital Government Research, 2005.

Digital Library

Google Scholar

[13]

H. Yang and J. Callan. Near-duplicate detection by instance-level constrained clustering. In Proceedings of the Twenty-Ninth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2006.

Digital Library

Google Scholar

[14]

S. Zelikovitz and H. Hirsh. Improving short text classification using unlabeled background knowledge. In P. Langley, editor, Proceedings of ICML-00, 17th International Conference on Machine Learning, pages 1183--1190, Stanford, US, 2000. Morgan Kaufmann Publishers, San Francisco, US.

Digital Library

Google Scholar

Cited By

View all

Carvalho NLourenço R(2018)E-RulemakingInternational Journal of Technology and Human Interaction10.4018/IJTHI.201804010314:2(35-53)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.4018/IJTHI.2018040103
Ghosh DChun SShafiq BAdam N(2016)Big Data-based Smart City PlatformProceedings of the 17th International Digital Government Research Conference on Digital Government Research10.1145/2912160.2912205(58-66)Online publication date: 8-Jun-2016
https://dl.acm.org/doi/10.1145/2912160.2912205
Hagen LHarrison TUzuner ÖFake TLamanna DKotfila CMossberger KHelbig NZhang JKim Y(2015)Introducing textual analysis tools for policy informaticsProceedings of the 16th Annual International Conference on Digital Government Research10.1145/2757401.2757421(10-19)Online publication date: 27-May-2015
https://dl.acm.org/doi/10.1145/2757401.2757421
Show More Cited By

Recommendations

E-Rulemaking: Lessons from the Literature

This article aims to identify the different technologies used in e-rulemaking platforms and initiatives and to assess the results from those initiatives. It aims to contribute to aid public officials responsible for the organization of such initiatives ...
Active learning for e-rulemaking: public comment categorization
dg.o '08: Proceedings of the 2008 international conference on Digital government research

We address the e-rulemaking problem of reducing the manual labor required to analyze public comment sets. In current and previous work, for example, text categorization techniques have been used to speed up the comment analysis phase of e-rulemaking --- ...
Electronic rulemaking: a public participation research agenda for the social sciences

This article proposes a social science research agenda that will reflect on and inform the development of new information technology-based approaches to the electronic collection, distribution, synthesis. and analysis of public commentary in the ...

Comments

Information & Contributors

Information

Published In

dg.o '08: Proceedings of the 2008 international conference on Digital government research

May 2008

488 pages

ISBN:9781605580999

Conference Chairs:
Monique Charbonneau
CEFRIO
,
Lester Diamond
US Social Security Administration
,
Stuart Shulman
University of Pittsburgh
,
Program Chairs:
Soon Ae Chun,
Marijn Janssen,
J. Ramon Gil-Garcia

Publisher

Digital Government Society of North America

Publication History

Published: 18 May 2008

Check for updates

Qualifiers

Research-article

Conference

dg.o '08

Sponsor:

NCDG

dg.o '08: Digital government research

May 18 - 21, 2008

Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 150 of 271 submissions, 55%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
114
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Carvalho NLourenço R(2018)E-RulemakingInternational Journal of Technology and Human Interaction10.4018/IJTHI.201804010314:2(35-53)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.4018/IJTHI.2018040103
Ghosh DChun SShafiq BAdam N(2016)Big Data-based Smart City PlatformProceedings of the 17th International Digital Government Research Conference on Digital Government Research10.1145/2912160.2912205(58-66)Online publication date: 8-Jun-2016
https://dl.acm.org/doi/10.1145/2912160.2912205
Hagen LHarrison TUzuner ÖFake TLamanna DKotfila CMossberger KHelbig NZhang JKim Y(2015)Introducing textual analysis tools for policy informaticsProceedings of the 16th Annual International Conference on Digital Government Research10.1145/2757401.2757421(10-19)Online publication date: 27-May-2015
https://dl.acm.org/doi/10.1145/2757401.2757421
Conrad AWiebe JHwa a(2012)Recognizing arguing subjectivity and argument tagsProceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics10.5555/2392701.2392711(80-88)Online publication date: 13-Jul-2012
https://dl.acm.org/doi/10.5555/2392701.2392711
Purpura SCardie CSimons JCharbonneau MDiamond LShulman SChun SJanssen MGil-Garcia J(2008)Active learning for e-rulemakingProceedings of the 2008 international conference on Digital government research10.5555/1367832.1367873(234-243)Online publication date: 18-May-2008
https://dl.acm.org/doi/10.5555/1367832.1367873

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Cited By

Index Terms

Recommendations

E-Rulemaking: Lessons from the Literature

Active learning for e-rulemaking: public comment categorization

Electronic rulemaking: a public participation research agenda for the social sciences