Publications

In Proceedings of the 32nd European Conference on Information Retrieval

In International Journal of Approximate Reasoning

In Encyclopedia of Database Systems

In Proceedings of the Second ACM International Conference on Web Search and Data Mining

In ECIR

In ACM Transactions On Information Systems

In information retrieval research, comparing retrieval approaches requires test collections consisting of documents, user requests and relevance assessments. Obtaining relevance assessments that are as sound and complete as possible is crucial for the comparison of retrieval approaches. In XML retrieval, the problem of obtaining sound and complete relevance assessments is further complicated by the structural relationships between retrieval results. A major difference between XML retrieval and flat document retrieval is that the relevance of elements (the retrievable units) is not independent of that of related elements. This has major consequences for the gathering of relevance assessments. This paper describes investigations into the creation of sound and complete relevance assessments for the evaluation of content-oriented XML retrieval as carried out at INEX, the evaluation campaign for XML retrieval. The campaign, now in its seventh year, has had three substantially different approaches to gather assessments and has finally settled on a highlighting method for marking relevant passages within documents - even though the objective is to collect assessments at element level. The different methods of gathering assessments at INEX are discussed and contrasted. The highlighting method is shown to be the most reliable of the methods.

In Proceedings of the Third Quantum Interaction Symposium

In Proceeedings of the 2nd International Conference on the Theory of Information Retrieval

In ArXiv e-prints

In SIGIR 2008

In Fourteenth String Processing and Information Retrieval Symposium (SPIRE 2007)

In Query Log Analysis: Social and Technological Challenges

Traditional search engine evaluation relies on a list of query document pairs along with a score reflecting the document relevance to the query. The score is generally a human assessment, but nothing is said explicitly about the actual user behavior. In this paper we illustrate with a toy model that once the user behavior is agreed upon, the human assessment can be eliminated and the engine performance can be evaluated based on the clickthrough data of past users.

In The 13th International Conference on Artificial Intelligence in Education (AIED 2007)

In 1st International Conference on the Theory of Information Retrieval

In Proceedings of the Sixteenth Conference on Information and Knowledge Management (CIKM 2007)

Web search engines consistently collect information about users interaction with the system: they record the query they issued, the URL of presented and selected documents along with their ranking. This information is very valuable: It is a poll over millions of users on the most various topics and it has been used in many ways to mine users interests and preferences. Query logs have the potential to partially alleviate the search engines from thousand of searches by providing a way to predict answers for a subset of queries and users without knowing the content of a document. Even if the predicted result is at rank one, this analysis might be of interest: If there is enough confidence on a user's click, we might redirect the user directly to the page whose link would be clicked. In this paper, we present three different models for predicting user clicks, ranging from most specific ones (using only past user history for the query) to very general ones (aggregating data over all users for a given query). The former model has a very high precision at low recall values, while the latter can achieve high recalls. We show that it is possible to combine the different models to predict with high accuracy (over 90%) a high subset of query sessions (24% of all the sessions).

In ACM Transactions On Information Systems

Standard Information Retrieval (IR) metrics are not well suited for new paradigms like XML or Web IR in which retrievable information units are document elements and/or sets of related documents. Part of the problem stems from the classical hypotheses on the user models: They do not take into account the structural or logical context of document elements or the possibility of navigation between units. This article proposes an explicit and formal user model that encompasses a large variety of user behaviors. Based on this model, we extend the probabilistic precision-recall metric to deal with the new IR paradigms.

In Semantics, Web and Mining, Joint International Workshops, EWMF 2005 and KDO 2005

In Proceedings of the 13th International Symposium on String Processing and Information Retrieval (SPIRE 2006)

In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Standard Information Retrieval (IR) metrics assume a simple model where documents are understood as independent units. Such an assumption is not adapted to new paradigms like XML or Web IR where retrievable informations are parts of documents or sets of related documents. Moreover, classical hypotheses assumes that the user ignores the structural or logical context of document elements and hence the possibility of navigation between units. EPRUM is a generalisation of Precision-Recall (PR) that aims at allowing the user to navigate or browse in the corpus structure. Like the Cumulated Gain metrics, it is able to handle continuous valued relevance. We apply and compare EPRUM in the context of XML Retrieval – a very active field for evaluation metrics. We also explain how EPRUM can be used in other IR paradigms.

In Proceedings of the 13th International Symposium on String Processing and Information Retrieval (SPIRE 2006)

In Information Retrieval

Most recent document standards like XML rely on structured representations. On the other hand, current information retrieval systems have been developed for flat document representations and cannot be easily extended to cope with more complex document types. The design of such systems is still an open problem. We present a new model for structured document retrieval which allows computing scores of document parts. This model is based on Bayesian networks whose conditional probabilities are learnt from a labelled collection of structured documents – which is composed of documents, queries and their associated assessments. Training these models is a complex machine learning task and is not standard. This is the focus of the paper: we propose here to train the structured Bayesian Network model using a cross-entropy training criterion. Results are presented on the INEX corpus of XML documents.

In XML and Information Retrieval workshop of SIGIR 2004

In XML retrieval paradigm, elements inside a document can be returned as answers to a user request. Since the information in an element is more specific than in a whole document, this might reduce the user effort in finding relevant information. However, as XML documents are composed of nested elements, many of which being possibly relevant to the user information need, retrieval systems must take care of the overlap issue before showing answers to the user. In this paper, we investigate how to disallow overlapping results by considering it as a filtering problem.

In Premiere COnference en Recherche d'Information et Applications (CORIA'04)

L'évaluation des systèmes de Recherche d'Information est depuis le début un des piliers de l'évolution de ce domaine. La qualité de l'évaluation est d'une importance capitale puisqu'elle permet de discriminer les différents modèles entre eux. Il est donc primordial de pouvoir constituer des corpus où les questions et leurs jugements de pertinence associés sont de qualité. Alors qu'avec des documents plats les méthodes sont bien établies, ce n'est plus le cas avec des documents structurés de type XML. Il est donc nécessaire de développer de nouvelle faç con d'évaluer. Nous présentons dans cet article l'interface utilisée lors de la campagne INEX 2003 qui permet d'évaluer de faç con la plus consistante et la plus exhaustive possible les documents XML.

In The First Twente Data Management Workshop

In this paper, we describe a new algebra for XML retrieval. We first describe how to transform an XPath-like query in our algebra. The latter contains a vague predicate, about, which defines a set of document parts within an XML document that fullfill a query expressed as in ``flat'' Information Retrieval - a query that contains only constraints on content but not on structure. This predicate is evaluated in a probabilistic way: we thus need a probabilistic interpretation of our algebra. Answers to query needs expressed with vague content and vague structure constraints can then be evaluated.

In Proceedings of the Thirteenth Conference on Information and Knowledge Management (CIKM 2004)

Comparing retrieval approaches requires test collections, which consist of documents, queries and relevance assessments. Obtaining consistent and exhaustive relevance assessments is crucial for the appropriate comparison of retrieval approaches. Whereas the evaluation methodology for flat text retrieval approaches is well established, the evaluation of XML retrieval approaches is a research issue. This is because XML documents are composed of nested components, which cannot be considered as independent in terms of relevance. This paper describes the methodology adopted in INEX (the INitiative for the Evaluation of XML Retrieval) to ensure consistent and exhaustive relevance assessments.

Dans le contexte de l'Accès à l'Information, la notion de document électronique a considérablement évoluée. En peu de temps, nous sommes passés d'un monde où la représentation dominante d'un document était constituée de la suite de ses mots ou de ses phrases à une représentation bien plus riche et structurée (multimédia). Cette évolution touche les communautés de la Recherche d'Information (RI), des Bases de Données et de l'Apprentissage Automatique qui sont celles qui sont au coeur de notre travail. Dans ce manuscrit, nous présentons un modèle complet de RI structurée basé sur les Réseaux Bayésiens (RB). Notre modèle est capable de répondre à des questions portant à la fois sur la structure et le contenu. Notre modèle peut également apprendre de manière automatique ses paramètres. Nous nous intéressons également à la définition d'une nouvelle mesure d'évaluation des systèmes de RI structurés.

In Machine Learning and Data Mining in Pattern Recognition

Most recent document standards rely on structured representations. On the other hand, current information retrieval systems have been developed for flat document representations and cannot be easily extended to cope with more complex document types. Only a few models have been proposed for handling structured documents, and the design of such systems is still an open problem. We present here a new model for structured document retrieval which allows to compute and to combine the scores of document parts. It is based on bayesian networks and allows for learning the model parameters in the presence of incomplete data. We present an application of this model for ad-hoc retrieval and evaluate its performances on a small structured collection. The model can also be extended to cope with other tasks such as interactive navigation in structured documents or corpus

In INitiative for the Evaluation of XML Retrieval (INEX). Proceedings of the Second INEX Workshop

This paper is the report of the working group on the evaluation assessment interface that was used in INEX'03. This paper describes the changes that are planned for INEX'04 and the different issues that were raised during the working group session.

In INitiative for the Evaluation of XML Retrieval (INEX). Proceedings of the Second INEX Workshop

We present a Bayesian framework for XML document retrieval. This framework allows us to consider content-only (CO) queries. We perform the retrieval task using inference in our network. The proposed model can adapt to a specific corpus through parameter learning and it uses a grammar to speed up the retrieval process in large or distributed databases. We also experimented list filtering to avoid overlap in the retrieved element list.

In Proceedings of INEX 2003

In INitiative for the Evaluation of XML Retrieval (INEX). Proceedings of the Second INEX Workshop

Since the 60's, evaluation has been a key problem for Information Retrieval (IR) systems and has been extensively discussed in the IR community. New IR paradigms, like Structured Information Retrieval (SIR), make classical evaluation measures inappropriate. A few tentative extensions to these measures has been proposed but are also inadequate. We do propose in this paper a new measure which is a generalisation of recall. This measure takes into account the specificity of SIR, when elements to be retrieved are linked by structural relationships. We show an instantiation of this measure on the INEX database and present experiments to show how well it is adapted to SIR evaluation.

In XML Workshop of the 25th ACM SIGIR Conference

Most recent document standards rely on structured representations. Nevertheless, current information retrieval systems have been developed for flat document representations and cannot be easily extended to cope with more complex document types. Only a few models have been proposed for handling structured documents, and the design of such systems is still an open problem. We present here a new model for structured document retrieval which allows to compute and to combine the scores of document parts. It is based on bayesian networks and makes use of machine learning algorithms for learning the model parameters in the presence of incomplete data. We present an application of this model for ad-hoc retrieval and evaluate its performances on a small structured collection.

In Proceedings of the First Annual Workshop of the Initiative for the Evaluation of XML retrieval (INEX)

We present a bayesian framework for XML document retrieval. This framework allows us to consider content only and content and structure queries. We perform the retrieval task using inference in our network. Our model can adapt to a specific corpora through parameter learning.

Avec l'émergence de nouveaux standards comme le XHTML ou le DocBook où la structure des documents est apparente, la communauté de recherche d'information a commencé à s'intéresser à l'utilisation de cette nouvelle source d'information. La tâche est ardue, car il s'agit de concilier de sources d'informations de natures différentes, à savoir le texte et la structure. Quelques modèles ont fait leur apparition ; mais ces travaux manquent encore de maturité et n'utilisent la structure que d'une manière simple. Le cadre théorique que nous présentons dans ce papier a pour vocation de permettre une prise en compte de la structure dans les tâches de recherche documentaire et de catégorisation. Ce modèle basé sur l'utilisation de réseaux bayésiens est capable de s'adapter à de nouvelles bases de données grâce à des techniques d'apprentissage numérique. Il offre également des perspectives de développement intéressantes comme par exemple la navigation interactive dans une base de données.

In Proceedings of the BCS-IRSG, 22nd Annual Colloquium on Information Retrieval Research

Since user's relevance judgments are a source of evidence for information retrieval, learning from this feedback is an appealing idea. Many different learning techniques have successfully been used for relevance feedback. In most models, learning is either performed off line or is based on simple heuristics. The approach we propose is based on a classical probabilistic model in which learning from feedback is simple and incremental. In this paper, we extend this model, presenting a new similarity function that takes into account feedback on the entire database while computing the score between a query and a single document. As a result, when a user judges a document it modifies the whole retrieval process.

In Colloque Francophone sur l'Apprentissage Automatique (CAP'2000)

Le but de la recherche documentaire (RD) consiste à trouver, parmi une base de documents, ceux qui répondent le mieux à une demande formulée par un utilisateur. Les expériences réalisées ont montré qu'il était impossible d'obtenir des résultats satisfaisants avec des systèmes où la représentation des documents ainsi que les paramètres utilisés lors de la recherche étaient figés. C'est pourquoi on emploie l'apprentissage pour modifier les paramètres du système de recherche, en se basant sur les jugements (feedback) que les utilisateurs peuvent porter sur la qualité des documents trouvés. Pourtant, l'application de telles techniques reste problématique. En effet, celles-ci sont soit complexes à mettre en oeuvre (traitement off-line), soit difficiles à contrôler. Nous proposons une approche est basée sur un modèle probabiliste de recherche documentaire qui permet d'utiliser le feedback de manière rapide et incrémentale. Dans cet article, nous étendons ce modèle pour ne plus regarder si un document répond à une requête dans l'absolu mais plutôt relativement à une base documentaire donnée. Ainsi, un jugement portant sur un seul document modifie l'ensemble du processus de recherche améliorant ainsi la rapidité de l'apprentissage. De plus, cette extension facilite notablement l'initialisation de la représentation des documents.