"Ranking sits at the core of information retrieval. Given a query, a collection of documents have to be ranked based on their relevance with respect to the query. Most modern search are based on learning to rank: given a training set composed of query-document pairs judged in terms of relevance, learn to rank documents given a query. Constructing training data for learning to rank is very expensive as it requires a significant human effort for judging the relevance of each document for each query. Search engines are used with very different queries run on very large document collections. Hence, it is impossible to judge each document in terms of relevance with respect to each query.The expense of constructing training data for learning to rank,and the need for different training data for different document collections/tasks is a major problem both for commercial companies and academic researchers. Given that most academic researchers do not have access to millions of dollars to create large scale learning to rank datasets, creating training data in an efficient and effective manner is crucial for enhancing the academic research on learning to rank. Hence, the primary objectives of this proposal are to: (1) build efficient and effective training data for learning to rank (2) increase the efficiency and effectiveness of learning to rank algorithms by devising objective metrics that can utilize the training data better, (3) develop techniques that can be used to adopt existing training data sets to the characteristics of different environments.The proposed techniques will allow researchers and engineers to develop better search systems with the same training data. This will directly affect the end users, enabling them to reach relevant information faster. The proposed methods can be used to create training datasets for a variety of different document collections/tasks, affecting the search experience of a broad set of users, from patent officers to medical doctors."