{Utrecht University}

LREC 2006 Workshop
Steven Krauwer

 {ELSNET}

Workshop
"Quality assurance and quality measurement for language and speech resources"
on Saturday, May 27th 2006
in conjunction with
LREC 2006, The 5th INTERNATIONAL CONFERENCE
ON LANGUAGE RESOURCES AND EVALUATION
Genoa, Italy, 24-26 May 2006


Programme

LREC 2006 QA
May 27 Start End Title Who
09:15 9:15 9:30 Introduction Steven Krauwer and Uwe Quasthoff
  9:30 10:20 What is quality Chris Cieri (invited talk)
  10:20 10:40 Validation of third party Spoken and Written Language Resources – Methods for performing Quick Quality Checks Hanne Fersøe, Henk van den Heuvel, Sussi Olsen
  10:40 11:00 Improving the Quality of FrameNet J. Scheffczyk, M. Ellsworth
11:00 11:00 11:30 BREAK 
  11:30 12:10 Valid Validations: Bare Basics and Proven Procedures
 Henk van den Heuvel (invited talk), in collaboration with Eric Sanders
  12:10 12:50 The Notion of Quality in Language Resources  -- Validation of the Spoken Dutch Corpus
 Hanne Fersøe (invited talk), in collaboration with Bart Jongejan and Sussi Olsen
  12:50 13:10 Quality control of treebanks: documenting, converting, patching Sabine Buchholz, Darren Green
  13:10 13:30 Evaluation of a diachronic text corpus Mikko Lounela
13:30 13:30 14:30 LUNCH 
  14:30 14:50 Measuring Monolinguality Uwe Quasthoff, Chris Biemann
  14:50 15:10 JTaCo & SProUTomat: Automatic Evaluation and Testing of Multilingual Language Technology Resources and Components Christian Bering and Ulrich Schäfer
  15:10 16:20 Panel session Chris Cieri (LDC), Chu-Ren Huang (Acad Sin.), Takenobu Tokunaga (TIT), Khalid Choukri (ELDA)
  16:20 16:30 Winding up & Closing
 Steven Krauwer and Uwe Quasthoff
16:30 16:30 17:00 BREAK 

Workshop description


The workshop aims at

It can be seen as a follow-up of the workshop on speech resources that took place at LREC 2004, but the scope is wider as we include both language and speech resources. We feel that there is a lot to be gained by bringing these communities together, if only because the speech community seems to have a longer tradition in resources evaluation than the written language community.

Relevance

Quality assurance is an important concern for both the provider, the distributor and the user of language and speech resources. The concept of quality is only meaningful if both the producer and the user of the resources can rely on the same set of quality criteria, and if there are effective procedures to check whether these criteria are met. The universe of possible types of language resources is huge and evolves over time, and there is no universal set of qualitative or quantitative criteria and tests that can be applied to all sorts of resources. In this workshop we will try to investigate what sorts of criteria, tests and measures are being used by providers, users and distribution agencies such as ELRA and LDC, and we will try to distill from this current practice general recommendations for quality assurance and measurement for language and speech resources, The workshop will look at quality assurance and quality measures both from the provider, the distributor and the user point of view, and will explicitly address special problems in connection with very large corpora, including numerical measures, comparison of corpora, exchange formats, etc.

Timetable


Submissions

Final versions of accepted papers are up to 6 pages long, and should be submitted in the format specified for the proceedings by the LREC organisers. The submission format is identical to the format required for the main conference. You can find the instructions at http://www.lrec-conf.org/lrec2006/article.php3?id_article=26

Workshop committee

Co-chairs:
Members:

Main contact and further info

Sponsors

This workshop is supported by ELSNET and WRITE (the international coordination committee for written language resources and evaluation).


Steven Krauwer (steven.krauwer@let.uu.nl) Utrecht Institute of Linguistics UiL OTS
Phone +31 30 253 6050; Fax +31 30 253 6000 Faculty of Humanities, Utrecht University
[Page last modified: 20-06-2008] Trans 10, 3512 JK Utrecht, Netherlands