Comparative Evaluation of Multilingual Information Access Systems: 4th Workshop of the Cross-Language Evaluation Forum, CLEF 2003, Trondheim, Norway, August 21-22, 2003, Revised Selected Papers

Julio Gonzalo, Carol Peters (auth.), Carol Peters, Julio Gonzalo, Martin Braschler, Michael Kluck (eds.)3540240179, 9783540240174, 9783540302223

The fourth campaign of the Cross-language Evaluation Forum (CLEF) for European languages was held from January to August 2003. Participation in this campaign showed a slight rise in the number of participants from the previous year, with 42 groups submitting results for one or more of the different tracks (compared with 37 in 2002), but a steep rise in the number of experiments attempted. A distinctive feature of CLEF 2003 was the number of new tracks and tasks that were offered as pilot experiments. The aim was to try out new ideas and to encourage the development of new evaluation methodologies, suited to the emerging requirements of both system developers and users with respect to today’s digital collections and to encourage work on many European languages rather than just those most widely used. CLEF is thus gradually pushing its participants towards the ultimate goal: the development of truly multilingual systems capable of processing collections in diverse media. The campaign culminated in a two-day workshop held in Trondheim, Norway, 21–22 August, immediately following the 7th European Conference on Digital Libraries (ECDL 2003), and attended by more than 70 researchers and system developers. The objective of the workshop was to bring together the groups that had participated in the CLEF 2003 campaign so that they could report on the results of their experiments.

---

Table of contents :
Front Matter….Pages –
Introduction….Pages 1-6
CLEF 2003 Methodology and Metrics….Pages 7-20
Analysis of the Reliability of the Multilingual Topic Set for the Cross Language Evaluation Forum….Pages 21-28
Evaluation of Information Access Technologies at the NTCIR Workshop….Pages 29-43
CLEF 2003 – Overview of Results….Pages 44-63
Report on CLEF-2003 Multilingual Tracks….Pages 64-73
The Impact of Word Normalization Methods and Merging Strategies on Multilingual IR….Pages 74-84
JHU/APL Experiments in Tokenization and Non-word Translation….Pages 85-97
Report on CLEF-2003 Experiments: Two Ways of Extracting Multilingual Resources from Corpora….Pages 98-107
Combining Query Translation and Document Translation in Cross-Language Retrieval….Pages 108-121
Cross-Language Experiments with the IR-n System….Pages 122-132
Multilingual Information Retrieval Using Open, Transparent Resources in CLEF 2003….Pages 133-139
ITC-irst at CLEF 2003: Monolingual, Bilingual, and Multilingual Information Retrieval….Pages 140-151
Language-Dependent and Language-Independent Approaches to Cross-Lingual Text Retrieval….Pages 152-165
Multilingual Retrieval Experiments with MIMOR at the University of Hildesheim….Pages 166-173
Concept-Based Searching and Merging for Multilingual Information Retrieval: First Experiments at CLEF 2003….Pages 174-184
CLEF 2003 Experiments at UB: Automatically Generated Phrases and Relevance Feedback for Improving CLIR….Pages 185-191
SINAI at CLEF 2003: Decompounding and Merging….Pages 192-201
Merging Results by Predicted Retrieval Effectiveness….Pages 202-209
MIRACLE Approaches to Multilingual Information Retrieval: A Baseline for Future Research….Pages 210-219
Experiments to Evaluate Probabilistic Models for Automatic Stemmer Generation and Query Word Translation….Pages 220-235
Clairvoyance CLEF-2003 Experiments….Pages 236-241
Simple Translations of Monolingual Queries Expanded Through an Association Thesaurus….Pages 242-252
Two-Stage Refinement of Query Translation in a Pivot Language Approach to Cross-Lingual Information Retrieval: An Experiment at CLEF 2003….Pages 253-262
Regular Sound Changes for Cross-Language Information Retrieval….Pages 263-270
Exeter at CLEF 2003: Experiments with Machine Translation for Monolingual, Bilingual and Multilingual Retrieval….Pages 271-285
Lexical and Algorithmic Stemming Compared for 9 European Languages with Hummingbird SearchServer TM at CLEF 2003….Pages 286-300
Océ at CLEF 2003….Pages 301-309
Comparing Weighting Models for Monolingual Information Retrieval….Pages 310-318
Pruning Texts with NLP and Expanding Queries with an Ontology: TagSearch….Pages 319-321
Report on CLEF-2003 Monolingual Tracks: Fusion of Probabilistic Models for Effective Monolingual Retrieval….Pages 322-336
Selective Compound Splitting of Swedish Queries for Boolean Combinations of Truncated Terms….Pages 337-344
COLE Experiments at CLEF 2003 in the Spanish Monolingual Track….Pages 345-357
Experiments with Self Organizing Maps in CLEF 2003….Pages 358-366
Ricoh at CLEF 2003….Pages 367-372
MediaLab at CLEF-2003: Using Keyword Disambiguation….Pages 373-375
The GIRT Data in the Evaluation of CLIR Systems – from 1997 Until 2003….Pages 376-390
A Data-Compression Approach to the Monolingual GIRT Task: An Agnostic Point of View….Pages 391-400
UC Berkeley at CLEF-2003 – Russian Language Experiments and Domain-Specific Retrieval….Pages 401-411
University of Hagen at CLEF 2003: Natural Language Access to the GIRT4 Data….Pages 412-424
The CLEF 2003 Interactive Track….Pages 425-434
iCLEF 2003 at Maryland: Translation Selection and Document Selection….Pages 435-449
UNED at iCLEF 2003: Searching Cross-Language Summaries….Pages 450-461
Comparing Syntactic-Semantic Patterns and Passages in Interactive Cross Language Information Access (iCLEF at University of Alicante)….Pages 462-467
Continued Experiments on Cross-Language Relevance Assessment….Pages 468-470
The Multiple Language Question Answering Track at CLEF 2003….Pages 471-486
Creating the DISEQuA Corpus: A Test Set for Multilingual Question Answering….Pages 487-500
Bridging Languages for Question Answering: DIOGENE at CLEF 2003….Pages 501-513
Cross-Language Question Answering at the USC Information Sciences Institute….Pages 514-522
How Frogs Built the Berlin Wall….Pages 523-534
Cross Lingual QA: A Modular Baseline in CLEF 2003….Pages 535-540
Question Answering in Spanish….Pages 541-548
Quantum, a French/English Cross-Language Question Answering System….Pages 549-558
A Cross–Language Question/Answering–System for German and English….Pages 559-571
Cross-Language French-English Question Answering Using the DLT System at CLEF 2003….Pages 572-580
The CLEF 2003 Cross Language Image Retrieval Track….Pages 581-593
Assessing Translation Quality for Cross Language Image Retrieval….Pages 594-610
Foreign Name Backward Transliteration in Chinese-English Cross-Language Image Retrieval….Pages 611-620
Image Retrieval: The MIRACLE Approach….Pages 621-630
Scene of Crime Information System: Playing at St. Andrews….Pages 631-645
The CLEF 2003 Cross-Language Spoken Document Retrieval Track….Pages 646-652
Exeter at CLEF 2003: Cross-Language Spoken Document Retrieval Experiments….Pages 653-657
N-Grams for Translation and Retrieval in CL-SDR….Pages 658-663
Spoken Document Retrieval Experiments with IR-n System….Pages 664-671
ITC-irst at CLEF 2003: Cross-Language Spoken Document Retrieval….Pages 672-675
Back Matter….Pages –