On perfect document rankings for expert search

Expert search systems often employ a document search component to identify on-topic documents, which are then used to identify people likely to have relevant expertise. This work investigates the impact of the retrieval effectiveness of the underlying document search component. It has been previously shown that applying techniques to the underlying document search component that normally improve the effectiveness of a document search engine also have a positive impact on the retrieval effectiveness of the expert search engine. In this work, we experiment with fictitious perfect document rankings, to attempt to identify an upper-bound in expert search system performance. Our surprising results infer that non-relevant documents can bring useful expertise evidence, and that removing these does not lead to an upper-bound in retrieval performance.