Optimizing recall/precision scores in Information retrieval over the WWW

Abstract

The rapid growth of the World Wide Web (WWW) and the massive size of the information corpus available for access symbolizes the wealth and benefits of this medium. At the same time this immense pool of information has created an information overflow which requires users to revert to techniques and tools in or- der to take advantage of such a resource and enhance the effectiveness of online information access. Search en- gines were created to assist users to find information by employing indexing techniques and suggest appropriate alternatives to browse. These search engines have in- efficiencies and are not focused enough to the needs of individual users and little has been done to ensure that the information presented is of a high recall and preci- sion standard. ‘Recall’ measures how efficient the system is at retrieving the relevant documents from the WWW, while ‘precision’ measures the relevance of the retrieved set of documents to the users’ requirements. We present our experiences with a system we devel- oped to optimize the recall/precision scores. We attempt to achieve this objective by employing a number of search engines and user profiling in tandem. Namely, we attempt to optimize: l recall by aggregating hits from major search engines and other previously developed retrieval systems, l precision by generating user profiles and predict- ing appropriate and focused information to specific users. Our system is able to easily and inexpensively accom- modate future generations of web-based retrieval systems and technologies. Our contribution to the IR field is that we were able to incorporate several desirable characteris- tics from different techniques to optimize and personalize WWW searching.