Georgeta Bordea
ABSTRACT
Traditionally, relevance assessments for expert search have been gathered through self-assessment or based on the opinions of co-workers. We introduce three benchmark datasets for expert search that use conference workshops for relevance assessment. Our data sets cover entire research domains as opposed to single institutions. In addition, they provide a larger number of topic-person associations and allow a more objective and fine-grained evaluation of expertise than existing data sets do. We present and discuss baseline results for a language modelling and a topic-centric approach to expert search. We find that the topic-centric approach achieves the best results on domain-specific datasets.
- P. Bailey, A. P. de Vries, N. Craswell, and I. Soboroff. Overview of the TREC 2007 Enterprise Track. In Proceedings of the Fourteenth Text REtrieval Conference (TREC), 2007.Google Scholar
- K. Balog, T. Bogers, L. Azzopardi, M. de Rijke, and A. van den Bosch. Broad Expertise Retrieval in Sparse Data Environments. In SIGIR '07: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 551--558, New York, NY, July 2007. ACM. Google ScholarDigital Library
- R. Berendsen, K. Balog, T. Bogers, A. Van den Bosch, and M. De Rijke. On the Assessment of Expertise Profiles. Journal of the American Society for Information Science, July 2013.Google ScholarCross Ref
- H. Biswas and M. Hasan. Using Publications and Domain Knowledge to Build Research Profiles: An Application in Automatic Reviewer Assignment. In Proceedings of the 2007 International Conference on Information and Communication Technology (ICICT'07), pages 82--86, 2007.Google ScholarCross Ref
- G. Bordea, S. Kirrane, P. Buitelaar, and B. O. Pereira. Expertise Mining for Enterprise Content Management. In LREC, pages 3495--3498, 2012.Google Scholar
- C. S. Campbell, P. P. Maglio, A. Cozzi, and B. Dom. Expertise identification using email communications. In CIKM '03: Proceedings of the Twelfth International Conference on Information and Knowledge Management, pages 528--531, New Orleans, LA, 2003. Google ScholarDigital Library
- H. Deng, I. King, and M. R. Lyu. Formal Models for Expert Finding on DBLP Bibliography Data. In ICDM '08: Proceedings of the Eighth IEEE International Conference on Data Mining, pages 163--172. IEEE, 2008. Google ScholarDigital Library
- S. T. Dumais and J. Nielsen. Automating the Assignment of Submitted Manuscripts to Reviewers. In SIGIR '92: Proceedings of the 15th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 233--244, New York, NY, USA, 1992. ACM. Google ScholarDigital Library
- S. Ferilli, N. Di Mauro, T. Basile, F. Esposito, and M. Biba. Automatic Topics Identification for Reviewer Assignment. Advances in Applied Artificial Intelligence, pages 721--730, 2006. Google ScholarDigital Library
- K. Hofmann, K. Balog, T. Bogers, and M. de Rijke. Contextual Factors for Finding Similar Experts. Journal of the American Society for Information Science, 61(5):994--1014, 2010. Google ScholarDigital Library
- M. Maybury. Expert finding systems. Technical Report MTR 06B000040, MITRE Corporation, 2006.Google Scholar
- D. Mimno and A. McCallum. Expertise Modeling for Matching Papers with Reviewers. In SIGKDD '07: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 500--509, 2007. Google ScholarDigital Library
- M. A. Rodriguez and J. Bollen. An Algorithm to Determine Peer-Reviewers. In '08: Proceedings of the Seventeenth International Conference on Information and Knowledge Management, pages 319--328. ACM, 2008. Google ScholarDigital Library
- E. Smirnova and K. Balog. A User-oriented Model for Expert Finding. In Proceedings of the 33rd European conference on Advances in information retrieval, ECIR'11, pages 580--592, Berlin, Heidelberg, 2011. Springer-Verlag. Google ScholarDigital Library
- I. Soboroff, A. P. de Vries, and N. Craswell. Overview of the TREC 2006 Enterprise Track. In In The Fifteenth Text Retrieval Conference (TREC 2006). NIST, 2006.Google Scholar
- M. Stankovic, J. Jovanovic, and P. Laublet. Linked Data Metrics for Flexible Expert Search on the Open Web. In Proceedings of the 8th extended semantic web conference on The semantic web: research and applications - Volume Part I, ESWC'11, pages 108--123, Berlin, Heidelberg, 2011. Springer-Verlag. Google ScholarDigital Library
- J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su. ArnetMiner: Extraction and Mining of Academic Social Networks. In KDD '08: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 990--998, New York, NY, USA, 2008. ACM. Google ScholarDigital Library
- D. Yarowsky and R. Florian. Taking the Load off the Conference Chairs: Towards a Digital Paper-Routing Assistant. In Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in NLP and Very Large Corpora, pages 220--230, 1999.Google Scholar
Index Terms
- Benchmarking domain-specific expert search using workshop program committees
Recommendations
Usefulness of click-through data in expert search
SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrievalThe task in expert finding is to identify members of an organisation with relevant expertise on a given topic. Typically, an expert search engine uses evidence from the authors of on-topic documents found in the organisation's intranet by search ...
Topic level expertise search over heterogeneous networks
In this paper, we present a topic level expertise search framework for heterogeneous networks. Different from the traditional Web search engines that perform retrieval and ranking at document level (or at object level), we investigate the problem of ...
Referral based expertise search system in a time evolving social network
COMPUTE '10: Proceedings of the Third Annual ACM Bangalore ConferenceTo solve some difficult problems that requires procedural knowledge, people often seek the advice of experts who have got competence in that problem domain. This paper focuses on locating and determining an expert in a particular knowledge domain. In ...
Comments