TrebleCLEF Logo
Evaluation, Best Practices and Collaboration for Multilingual Information Access
TrebleCLEF News
(Events | Evaluation Initiatives | Resources & Tools | Job Announcements | Miscellaneous)


CLEF 2010 (Posted on Monday, 19th April 2010)
Padua, Italy September 2010

TrebleCLEF workshop at eChallenges (Posted on Friday, 16th October 2009)
Best Practices for Multilingual Information Access Istanbul, Turkey 21 October 2009

MLIA Technology Transfer Day (Posted on Thursday, 8th October 2009)
Berlin, Germany 8 December 2009

Query Log Analysis: From Research to Best Practice (Posted on Tuesday, 28th April 2009)
London, UK
27-28 May 2009

Cross Language Evaluation Form (Posted on Saturday, 21st February 2009)
CLEF 2009 Call for Participation
Registration for CLEF 2009 is now open – see under CLEF 2009 – How to Participate

INFILE (Posted on Friday, 20th February 2009)
INFILE : Information, Filtering, Evaluation Evaluation campaign for information filtering systems

CLEF 2009 (Posted on Friday, 20th February 2009)
New Pilot Track: Grid@CLEF

ECIR Workshop on Information Retrieval over Social Networks (Posted on Friday, 20th February 2009)

Multimedia Grand Challenge @ ACM MM 2009 (Posted on Friday, 20th February 2009)

TrebleCLEF Summer School on Multilingual Information Access (Posted on Wednesday, 11th February 2009)
Santa Croce in Fossabanda
Pisa, Italy
15-19 June 2009

FLaReNet (Posted on Sunday, 8th February 2009)
Launching Event
Vienna, Austria, 12-13 February 2009

NEWS (Posted on Friday, 30th January 2009)
Named Entities Workshop: Shared Task on Transliteration (NEWS 2009 : An ACL-IJCNLP 2009 Workshop)

NTCIR 2008 (Posted on Tuesday, 13th January 2009)
The 7th NTCIR Workshop (2007/2008)
Evaluation of Information Access Technologies:
Information Retrieval, Question Answering, and and Cross-Lingual Information Access
Final Meeting: December 16-19, 2008, NII, Tokyo, Japan

ESSIR 09 (Posted on Monday, 12th January 2009)
Panel on Evaluation in Information Retrieval
at ESSIR 2009
Padua, Italy
31 August - 4 September 2009

CIRSE Workshop (Posted on Thursday, 8th January 2009)
ECIR 09 Workshop on Contextual Information Access, Seeking and Retrieval Evaluation

NAACL-HLT 2009 (Posted on Monday, 29th December 2008)
Third International Workshop on Cross Lingual Information Access: Addressing the Information Need of Multilingual Societies (CLIAWS3)
Boulder, Colorado; June 04,2009

SEPLN 2009 (Posted on Monday, 22nd December 2008)
25th Conference of the Spanish Society for Natural Language Processing
Palacio Miramar, Donostia - San Sebastián, 8-10 September 2009

ECIR 2009 (Posted on Friday, 19th December 2008)
Workshop on Information Retrieval over Social Networks

ENQOIR 2009 (Posted on Monday, 15th December 2008)
First International Workshop on Aspects in Evaluating Holistic Quality of Ontology-based Information Retrieval
to be held in coonjunction with APWeb-WAIM 2009 conferences
Suzhou, China, 1-4 April 2009

ACL-IJCNLP 2009 (Posted on Monday, 15th December 2008)
Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing
Singapore, 2-7 August 2009

AIRS 2009 (Posted on Monday, 15th December 2008)
The Fifth Asia Information Retrieval Symposium (AIRS 2009)
Hokkaido University, Sapporo, Japan, 21-23 October 2009

KGCM 2009 (Posted on Wednesday, 26th November 2008)
III International Conference on Knowledge Generation, Communication and management
Orlando, Florida, USA, 10-13 July 2009

ACM CIVR 2009 (Posted on Wednesday, 26th November 2008)
ACM International Conference on Image and Video Retrieval
Santorini Island, Greece, 8-10 July, 2009

2nd ACM International Workshop (Posted on Thursday, 12th June 2008)
Improving Non English Web Searching: iNEWS 2008
held in conjunction with ACM CIKM 2008
Fotis Lazarinis, Efthimis Efthimiadis, Jesus Vilares, John Tait
October 30, 2008, Napa Valley, California


First Call: QUAERO/ImageCLEF Workshop on Multimedia Information Retrieval Evaluation (Posted on Tuesday, 13th May 2008)
ImageCLEF organises a pre-CLEF workshop on September 16, 2008 co-located with the CLEF Workshop, which is co-located with the ECDL 2008 in Aarhus, Denmark.
Currently we are in a very early planning stage. The workshop will be organised similarly to the preliminary MUSCLE/ImageCLEF workshops (
If you are interested in presenting a paper, please contact us directly.

11th Oriental COCOSDA Workshop (Posted on Tuesday, 6th May 2008)
Kyoto, Japan
25-27 November 2008
Web site Information will be loaded soon

Workshop Information Retrieval 2008 (WIR 2008) (Posted on Tuesday, 6th May 2008)
University of Würzburg, Germany
06-08 October 2008
Call for Papers, submission deadline 06 June 2008

Workshop on Novel Methodologies for Evaluation in Information retrieval (Posted on Friday, 14th March 2008)
ECIR 2008, Glasgow, UK, March 30, 2008

SIEDL 2008 - Tenerife, Spain, June 1, 2008 (Posted on Monday, 10th March 2008)
Submission deadline extended to March 17th, 2008

Evaluation Initiatives

EVALITA 2009 - CfP (Posted on Friday, 20th February 2009)
Evaluation of NLP and Speech Tools for Italian

EVALITA 2009 (Posted on Wednesday, 26th November 2008)
Evaluation of NLP Tools for Italian

FIRE 2008 (Posted on Tuesday, 11th November 2008)
First Workshop of the Forum for Information Retrieval Ebauation (FIRE 2008)

II Web People Search Evaluation Workshop (Posted on Wednesday, 8th October 2008)
Finding information about people in the World Wide Web is one of the most common activities of Internet users. Person names, however, are highly ambiguous.

4th Textual Entailment Challenge at TAC 2008 (Posted on Tuesday, 3rd June 2008)
Second Announcement
Extended Registration Deadline: 27 June 2008


Setooz Search Engine (Posted on Thursday, 29th January 2009)
A research group at IIIT Hyderabad is developing a multilingual search engine: SETOOZ:

Okapi IR (Posted on Tuesday, 23rd December 2008)
Experimental Okapi IR as Open Source software, under the BSD license

Tutorial Image Retrieval (Posted on Thursday, 18th September 2008)
Tutorial Image Retrieval @ ICPR 2008- ICPR 2008 Tampa, Florida, USA 7 December 2008

JRC-Acquis and DGT Translation Memory (DGT-TM) (Posted on Tuesday, 3rd June 2008)
This is an update regarding the two freely available multilingual resources JRC-Acquis and DGT Translation Memory (DGT-TM), distributed by the European Commission’s Joint Research Centre (JRC):

(1) For version 3.0 of the multilingual parallel corpus JRC-Acquis (22 languages, 231 language pairs; 1 Billion words), bilingual alignments have now also been produced with the alignment tool HunAlign. Until recently, only Vanilla alignments were available. This additional resource will allow comparison and benchmarking of alignment software for a wide variety of language pairs, and users can choose the alignments that suit them better.

(2) Following frequent user requests, the tool to extract bilingual translation memories for all 231 DGT-TM language pairs is now available as Java byte code so that the extraction tool can be used on operating systems other than Windows.

For more information on both resources, see and

New Demo Versions of the MultiMatch (Multilingual Multimedia) Search Engine (Posted on Wednesday, 21st May 2008)
can be viewed at

TrebleCLEF & IRF (Posted on Friday, 9th May 2008)
The International Retrieval Facility is offering access to their computing facilities for CLEF participants. For details see

CLEF goes non-European (Posted on Friday, 29th February 2008)
For the first time this year CLEF offers target collections for non-European languages: Persian in the Ad-Hoc track and Arabic in INFILE. For more information, see CLEF 2008 Agenda (

EDLNet: Release of Europeana Maquette (Posted on Friday, 22nd February 2008)
The demo or maquette of the Europeana site is now live. Please see link below
Press Release

CMU makes available OpenEphyra open source QA system (Posted on Friday, 22nd February 2008)
The question answering research group at CMU has decided to place the Ephyra QA system into open source.

EDLNet: Vision on Multilinguality (Posted on Thursday, 21st February 2008)
Presented at EDL Project Conference
Deutsch Nationalbibliothek, Frankfurt am Main,

Special Issue on "Non-English Web Retrieval" (Posted on Wednesday, 20th February 2008)
Springer Academic Publishers
Journal of Information Retrieval
Important dates:
Abstract submission: May 4, 2008
Paper submission: May 11, 2008, 11:50pm Hawaii Standand Time
Notification of acceptance/rejection: June 25, 2008

Job Announcements

three-year position in multilingual text analysis (Posted on Thursday, 19th February 2009)
The European Commission’s Joint Research Centre (JRC) in Ispra, at the Lago Maggiore in Northern Italy, has an opening for a three-year position in multilingual text analysis. Applicants will either need to have completed a Ph.D. or have five years of relevant post-graduate experience.

Job Announcement (Posted on Wednesday, 26th November 2008)
PhD position in Machine Learning for Multimedia Information Retrieval

Job Announcement (Posted on Friday, 10th October 2008)
Research scientist position
MAVIR Consortium, Madrid, Spain
Multimedia Information Retrieval Project


Job Announcement (Posted on Tuesday, 19th August 2008)
University of Glasgow Seeks Director for HATII

Job Announcement (Posted on Tuesday, 19th August 2008)
1 Postdoctoral research position -- deadline 15 September 2008
Applications are invited for a postdoctoral research position in Computational Text Mining for a 5 years contract. Selected candidates will work on the development computational techniques to analyse huge comparable and/or parallel corpora either on syntactic and semantic clustering and classification of single- and multi-word terms, or on extraction of their translations. Supervised classification of documents is a transversal problem that may be also necessary to tackle. The position is funded under the "Commitment to Science" program, launched by the Portuguese National Foundation for Science and Technology. The program aims to provide fixed duration work contracts between the researchers and the host institution.
More info at

Job Announcement (Posted on Monday, 18th August 2008)
Localisation is the adaptation of digital content to culture, locale and linguistic environment. The Centre for Next Generation Localisation (CNGL) is a large Academia-Industry partnership, funded by Science Foundation Ireland and Industry Partners, with over 100 researchers developing novel technologies addressing the key localisation challenges of volume, access and personalisation.

The major research strands within the CNGL are Integrated Language Technologies (ILT), Digital Content Management (DCM), Localisation Technologies and Processes (LOC) and Systems Framework (SF). We are currently recruiting:

Post-Doctoral Research Positions:

3 Post-Doctoral Positions in ILT (MT, NLP)
1 Post-Doctoral Position in DCM (Ontology Induction)
3 Post-Doctoral Positions in LOC (Workflow, Translation, Multilingual Content)
Post-Doctoral positions are for 3 years (1 year contract initially). Salary: €38,623-45,401 per annum (depending on experience). Starting dates: now – November 2008.

PhD Studentship Research Positions:

5 PhD Studentships in ILT (MT, NLP)
5 PhD Studentships in DCM (IR/IE, QA, Ontology Induction)
8 PhD Studentships in LOC (Workflow, Translation, Multilingual Content)
PhD positions are typically for 4 years. Stipend: €16,000 (tax free) plus payment of registration fees. Starting dates: now – November 2008.

CNGL provides state-of-the-art research facilities and supports travel to present at conferences. Please visit for more detailed information on each position.

The successful candidates will join well established research groups at Dublin City University, Trinity College Dublin, University College Dublin and University of Limerick, Ireland.
Deadline for applications: 31st August 2008

To apply send CV and contact details of 2 referees to quoting the appropriate reference (see Please also use for informal inquiries.

Job Announcement (Posted on Monday, 26th May 2008)
The Human Language Technologies research unit of Fondazione Bruno Kessler (Italy) offers, in collaboration with the University of Trento (Italy), grants for doctoral students in the following research areas:
- 1 fellowship in Automatic Speech Recognition, (contact: Diego Giuliani
- 1 fellowship in Spoken and Written Language Translation (contact: Marcello Federico
- 2 fellowships Content Processing (contact: Bernardo Magnini

Candidates should meet the following requirements:
- Degree in Computer Science, Electrical Engineering, Physics or Mathematics.
- Studies should have been completed efficiently, recently and with an above of average mark.
- Advanced knowledge in either Computational Linguistics, Machine Learning, Statistics, Patter Recognition, or Speech Recognition.
- Programming experience in C/C++ and/or Java, Python, Perl in the Unix environment.
- Good English knowledge.

Doctorate fellowships, which are sponsorate by FBK, will be formally pursued at the ICT International Doctorate School of the University of Trento. The program starts in November 2008, has a minimum length of three years and includes attendance of courses during the first two years, while the third is fully dedicated to research work. Detailed information about the School and the grant assigned to students can be found in

Interested candidates are invited to submit their application through the website of the ICT School and to get in touch with the above given contact points to know more about the proposed research topics. Information about research activities at FBK and opportunities of pre-enrollment internships can be found in

Closing date for applications: May 2008

Job Announcement (Posted on Monday, 28th April 2008)
post-doc position in multilingual text analysis

Job Announcement (Posted on Tuesday, 15th April 2008)
Open Research Positions: Centre for Next Generation Localisation (CNGL), Dubli, Ireland


The quality of legal dictionaries: an assesment (Posted on Thursday, 29th January 2009)
De Groot & van Laer “The quality of legal dictionaries: an assesment” (full text on (pdf); abstract on ) [Maastricht Faculty of Law working paper & Social

FLaReNet (Posted on Monday, 29th December 2008)
Announcing FLaReNet (Fostering Language Resources Network)