Applying Diversity and Novelty to Personalized Search

Abstract: Recent research into retrieving and reordering search results so that they cover a maximum number of topics and information needs has gained traction as a means to assist navigation through the explosion of available online information. This problem is also addressed by personalized search: the retrieving and reordering of search results biased to the preferences of a particular user. We investigate a search personalization system that builds a user model from the latent topics mined from their queries and clicked documents. We find that a user’s clicked documents exist in a specific area of the latent topic space. By “diversifying” search results biased to our user model we can reorder search results to the user’s preferences.

Above is a scatter plot of the location of each query and document the user clicked on for that query in topic space. We see that the user appears to occupy a distinguished manifold in 3-topic space.

All code written in R and Ruby.  Download the personalize search topics paper.