Cross-language text retrieval by query translation using term reweighting

Abstract

In a dictionary-based query translation for cross-language text retrieval, transfer ambiguity is one of the main causes of performance deterioration, but the problem has not received significant attention in this field. To resolve transfer ambiguity, this paper proposes a two-phase query translation based on term reweighting, which uses a bilingual transfer dictionary, originally designed for machine translation. In general, source language query terms each show some word association with others, so that their correct translations should be more likely to co-occur in target documents. Based on this simple intuition, the first phase discriminates more relevant target documents from the others. Using statistical and ranking information from the highly relevant documents, the second phase then converts a translated query vector into reweighted form to add an extra weight on probably correct target terms. In experiments, the results were remarkable: the proposed method achieved almost the same performance as the monolingual IR system, actually contributing to an improvement of precision by about 9% over a baseline system.