CoNLL Shared Task 2015: Call for Proposals

SIGNLL (ACL's Special Interest Group on Natural Language Learning)
invites proposals for the CoNLL Shared Task 2015. Since 1999, CoNLL
(the Conference on Computational Natural Language Learning) has
included a shared task in which training and test data are provided
by the organizers, which allows participating systems to be evaluated
and compared in a systematic way. Descriptions of the participating
systems and an evaluation of their performance are presented both at
the conference and in the proceedings. Previous shared tasks have
been devoted to chunking (1999, 2000), clausing (2001), named entity
recognition (2002, 2003), semantic role labeling (2004, 2005),
dependency parsing (2006, 2007), joint parsing of syntactic and
semantic dependencies (2008, 2009), hedge detection (2010),
co-reference resolution (2011, 2012), and grammatical error 
correction (2013, 2014). For CoNLL-2015 proposals are invited for 
any shared task involving natural language learning, including not 
only natural language processing tasks and applications but also 
the modeling of human language learning.

Proposals should contain the following information:

- Names and affiliations of the members of the organizing committee
  (including a designated contact person)
- Description of the shared task, including its relevance for the
  study of computational natural language learning
- Description of the process for selecting, acquiring and preparing the
  data for training and evaluation, including conditions of 
  availability
- Description of the procedure for evaluating the performance of
  participating systems, including the availability of evaluation
  software
- Any other relevant information

A tentative timeline for the CoNLL Shared Task 2015 is as follows:

- November 15, 2014: general announcement (made by SIGNLL)
- December 17, 2014: call for interest in participation
- February 1, 2015: task website (first version with general 
  information, task definition, examples and calendar)
- February 1, 2015: trial datasets (plus documentation and scorer)
- February 15, 2015: registration for the task (plus corpora licenses)
- March 1, 2015: training and development sets available
- May 5, 2015: test set available
- May 10, 2013: systems' outputs collected

Proposals should make clear that a similar schedule can be 
implemented, especially for the preparation of resources 
(trial/training/development/test data sets).

The deadline to submit a proposal is *** September 15, 2014 ***. 
Proposals should be in PDF format, 2 to 3 pages long, even though 
we encourage proposers to include annexes with examples that help 
understanding the data and type of evaluation. Proposals should 
be submitted by e-mail to proposals@signll.org, from 
the email of the contact person, with email subject “CoNLL-2015 
Shared Task Proposal”. Proposals will be reviewed by the SIGNLL 
steering committee and proposers will be notified about the 
selection before October 31. Proposers might be contacted 
during the reviewing period to provide further information.

We ask potential proposers to indicate the intention to 
submit by emailing proposals@signll.org. The 
same email address can be used for questions about the 
process, or to solicit early feedback for a proposal.