The 9th Workshop on Innovative Use of NLP for Building Educational Applications

Pre-Workshop Information

For those attending the BEA9 Workshop this Thursday, June 26, there are a few notes:

• Proceedings: We encourage everyone to make use of the free download of the entire ACL proceedings before coming to the conference. It includes the proceedings for the main conference as well as the satellite events, including the BEA9 Workshop. The proceedings for the BEA9 can be found in the ACL Anthology.

• Poster Session: This year we are having two back-to-back poster sessions which will allow people to better circulate and see more posters. Each session is 45 minutes and consists of 7 papres each. Please see the workshop program below to see which poster is in which session.

• Oral Presentations: For those presenting orally, please send your presentation to bea.nlp.workshop@gmail.com in advance or come early to the workshop between 8:45 - 9:00 AM so we can load the presentations on one machine to make the transitions between talks more seamless.

• BEA9 T-shirts: This year is the second year we've had sponsors and we are using a portion of the sponsorship to fund our first ever workshop t-shirts. T-shirts are available for all who register. We will be distributing the t-shirts at the workshop on Thursday. Please make sure you have proof of registration with you. We ordered a range of sizes but if you want the best chance of getting a size that fits you, please come to the workshop early on Thursday. There is one t-shirt per registrant.

• Post-Workshop Dinner: As in previous editions, we will once again be hosting our "world famous" post-workshop dinner. This year's will be at 7:00 at Darbar: Cuisine of India, located at 1911 Aliceanna Street, a nice 15 minute walk from the conference hotel along the water. Directions can be found here. In the morning of the workshop, we will be getting a head count to know how many seats to reserve, so please let us know then or by email beforehand. Finally, we will be using the remaining sponsorship funds to subsidize the dinner for students.

Workshop Description

The field of NLP and education has dramatically matured since the first BEA workshop in 1997 , where the primary focus was on grammatical error detection. As a community, we have continued to improve existing capabilities and to identify and develop innovative and creative NLP approaches for use in educational settings. In the writing domain, automated writing evaluation systems are now commercially viable, and are used to score millions of test-taker essays on high-stakes assessments. In speech, major advances in speech technology, have made it possible to include speech in both assessment and Intelligent Tutoring Systems (ITS). Spoken constructed responses are now being used in low-stakes and practice applications. Consistent with this, there is also a renewed interest in spoken dialog for instruction and assessment. Relative to continued innovation, the explosive growth of mobile applications has increased interest in game-based applications for instruction and assessment. The current educational and assessment landscape, especially in the United States, continues to foster a strong interest and high demand that pushes the state-of-the-art in automated writing evaluation capabilities to expand the analysis of written responses to writing genres other than those presently found in standardized assessments. Much of the current demand for creative, new educational applications stems from the development of the Common Core State Standards Initiative (CCSSI). CCSSI describes what K-12 students should be learning with regard to reading, writing, speaking, listening, language, and media and technology. The goal of CCSSI is to ensure college- and workplace-readiness across those domains.

In the past few years, the use of NLP in educational applications has gained visibility outside of the computational linguistics (CL) community. First, the Hewlett Foundation reached out to public and private sectors and sponsored two competitions (both inspired by the CCSSI): one for automated essay scoring, and the other for scoring of short response items. The motivation driving these competitions was to engage the larger scientific community in this enterprise. Massive Open Online Courses (MOOCs) are now also beginning to incorporate automated writing evaluation systems to manage the thousands of assignments that may be received during a single MOOC course (New York Times). Another breakthrough for educational applications within the CL community is the presence of a number of shared-task competitions over the last three years. There have been three shared tasks on grammatical error correction with the most recent edition hosted at CoNLL 2013. Also in 2013 there was a SemEval Shared Task on Student Response Analysis and one on Native Language Identification (hosted at the 2013 edition of this workshop). All of these competitions increased the visibility of the research space for NLP for building educational applications. While attendance has continued to be strong for several years, 2013 was a banner year for the BEA workshop as it was the largest ever and had the largest attendance count of any one-day workshop at NAACL.

The 2014 workshop will solicit both full papers and short papers for either oral or poster presentation. Given the broad scope of the workshop, we organize the workshop around three central themes in the educational infrastructure: (1) development of curriculum and assessments; (2) delivery of curriculum and assessments; and (3) reporting of assessment outcomes. In the 2014 workshop, we will solicit papers for educational applications that incorporate NLP methods, including, but not limited to: automated scoring of open-ended textual and spoken responses; game-based instruction and assessment; intelligent tutoring; grammatical error detection; learner cognition; spoken dialog; tools for teachers and test developers; and use of corpora. Research that incorporates NLP methods for use with mobile and game-based platforms, and academic ePortfolio sytems or MOOCs would be of special interest. Since the first workshop in 1997, the BEA workshop series has continued to bring together many NLP subfields, and to foster interaction and collaboration among researchers in academia and industry. The workshop offers a unique venue for researchers to present and discuss their work. Each year, we see steady growth in workshop submissions and attendance, and the research has become more advanced. In 2014, we expect that the workshop (consistent with the eight previous workshops at ACL and NAACL/HLT), will continue to expose the NLP research community to technologies that identify novel opportunities for the use of NLP techniques and tools in educational applications. Topics will include, but will not be limited to, the following:

• Automated scoring/evaluation for written student responses
  • Content analysis for scoring/assessment
  • Analysis of the structure of argumentation
  • Grammatical error detection and correction
  • Discourse and stylistic analysis
  • Plagiarism detection
  • Machine translation for assessment, instruction and curriculum development
  • Detection of non-literal language (e.g., metaphor)
  • Sentiment analysis
  • Non-traditional genres (beyond essay scoring)
  
  
• Intelligent Tutoring (IT) and Game-based assessment that incorporates NLP
  • Dialogue systems in education
  • Hypothesis formation and testing
  • Multi-modal communication between students and computers
  • Generation of tutorial responses
  • Knowledge representation in learning systems
  • Concept visualization in learning systems
  
  
• Learner cognition
  • Assessment of learners' language and cognitive skill levels
  • Systems that detect and adapt to learners' cognitive or emotional states
  • Tools for learners with special needs
  
  
• Use of corpora in educational tools
  • Data mining of learner and other corpora for tool building
  • Annotation standards and schemas / annotator agreement
  
  
• Tools and applications for classroom teachers and/or test developers
  • NLP tools for second and foreign language learners
  • Semantic-based access to instructional materials to identify appropriate texts
  • Tools that automatically generate test questions
  • Processing of and access to lecture materials across topics and genres
  • Adaptation of instructional text to individual learners' grade levels
  • Tools for text-based curriculum development
  • E-learning tools for personalized course content
  • Language-based educational games
  
  
• Descriptions and proposals for shared tasks

Invited Speaker

We are happy to have Dr. Norbert Elliot as our invited speaker for the BEA9! Here is the title, abstract and bio.

Submission Information

We will be using the ACL 2014 Submission Guidelines for the BEA9 Workshop this year. Authors are invited to submit a full paper of up to 9 pages of content with up to 2 additional pages for references. We also invite short papers of up to 5 pages of content, including 2 additional pages for references. Please note that unlike previous years, final, camera ready versions of accepted papers will not be given an additional page to address reviewer comments.

Papers which describe systems are also invited to give a demo of their system. If you would like to present a demo in addition to presenting the paper, please make sure to select either "full paper + demo" or "short paper + demo" under "Submission Category" in the START submission page.

Previously published papers cannot be accepted. The submissions will be reviewed by the program committee. As reviewing will be blind, please ensure that papers are anonymous. Self-references that reveal the author's identity, e.g., "We previously showed (Smith, 1991) ...", should be avoided. Instead, use citations such as "Smith previously showed (Smith, 1991) ...".

Please use the 2014 ACL style sheets for composing your paper: http://www.cs.jhu.edu/ACL2014/CallforPapers.htm (see "ACL 2014 Style Files" section).

We will be using the START conference system to manage submissions: https://www.softconf.com/acl2014/BEA9

Important Dates

• Submission Deadline: March 25 - 23:59 EST (New York City Time) [ Current EST ]
• Notification of Acceptance: April 11
• Camera-ready Papers Due: April 28
• Workshop: June 26

Presentation Information

Oral Presentations: Long papers accepted for oral presentations are allotted 20 minutes for the talk and 5 minutes for questions. Short papers that are accepted for oral presentations are allotted 15 minutes for the talk and 5 minutes for questions.

Poster Presentations: All papers accepted for a poster presentation will be presented in the session after lunch between 2:00 and 3:30. The posterboards will be self-standing, on top of tables (giving room for laptops, business cards, handouts, etc). The posterboards measure 36 inches high and 48 inches wide. Double-sided tape, pushpins, etc. for affixing the posters to the boards will be provided.

Workshop Program

Program Committee

• Andrea Abel, EURAC, Italy
• Oistein Andersen, University of Cambridge, UK
• Sumit Basu, Microsoft Research, USA
• Timo Baumann, University of Hamburg, Germany
• Lee Becker, Hapara, USA
• Delphine Bernhard, Université de Strasbourg, France
• Jared Bernstein, Pearson, USA
• Kristy Boyer, North Carolina State University, USA
• Chris Brew, Nuance Communications, Inc., USA
• Ted Briscoe, University of Cambridge, UK
• Chris Brockett, Microsoft Research, USA
• Julian Brooke, University of Toronto, USA
• Aoife Cahill, Educational Testing Service, USA
• Min Chi, North Carolina State University, USA
• Martin Chodorow, Hunter College, CUNY, USA
• Mark Core, University of Southern California, USA
• Daniel Dahlmeier, SAP, Singapore
• Barbara Di Eugenio, University of Illinois at Chicago, USA
• Markus Dickinson, Indiana University, USA
• Bill Dolan, Microsoft Research, USA
• Myrosia Dzikovska, University of Edinburgh, UK
• Yo Ehara, Miyao Lab., National Institute of Informatics, Japan
• Maxine Eskenazi, Carnegie Mellon University, USA
• Keelan Evanini, ETS, USA
• Michael Flor, ETS, USA
• Peter Foltz, Pearson Knowledge Technologies, USA
• Jennifer Foster, Dublin City University, Ireland
• Thomas Francois, UC Louvain, Belgium
• Anette Frank, University of Heidelberg, Germany
• Michael Gamon, Microsoft Research, USA
• Caroline Gasperin, Swiftkey, UK
• Kallirroi Georgila, University of Southern California
• Iryna Gurevych, University of Darmstadt, Germany
• Na-Rae Han, University of Pittsburgh, USA
• Trude Heift, Simon Frasier University, Canada
• Michael Heilman, ETS, USA
• Derrick Higgins, ETS, USA
• Rebecca Hwa, University of Pittsburgh, USA
• Radu Ionescu, University of Bucharest, Romania
• Ross Israel, Indiana University, USA
• Pamela Jordan, University of Pittsburgh, USA
• Levi King, Indiana University, USA
• Ola Knutsson, Stockholm University, Sweden
• Ekaterina Kochmar, University of Cambridge, UK
• Mamoru Komachi, Tokyo Metropolitan University, Japan
• John Lee, City University of Hong Kong
• Baoli Li, Henan University of Technology, China
• Diane Litman, University of Pittsburgh, USA
• Annie Louis, University of Edinburgh, UK
• Xiaofei Lu, Penn State University, USA
• Nitin Madnani, ETS, USA
• Montse Maritxalar, University of the Basque Country, Spain
• Mourad Mars, University of Monastir, Tunisia
• James Martin, University of Colorado, USA
• Aurélien Max, LIMSI-CNRS, France
• Julie Medero, University of Washington, USA
• Detmar Meurers, University of Tubingen, Germany
• Lisa Michaud, Merrimack College, USA
• Rada Mihalcea, University of Michigan, USA
• Michael Mohler, Language Computer Corporation, USA
• Jack Mostow, Carnegie Mellon University, USA
• Smaranda Muresan, Columbia University, USA
• Ani Nenkova, University of Pennsylvania, USA
• Hwee Tou Ng, National University of Singapore, Singapore
• Rodney Nielsen, University of Colorado, USA
• Mari Ostendorf, University of Washington, USA
• Ted Pedersen, University of Minnesota, USA
• Heather Pon-Barry, Arizona State University, USA
• Matt Post, Johns Hopkins University, USA
• Patti Price, PPRICE Speech and Language Technology, USA
• Marti Quixal, University of Texas at Austin, USA
• Carolyn Rosé, Carnegie Mellon University, USA
• Andrew Rosenberg, Queens College, CUNY, USA
• Mihai Rotaru, TextKernel, the Netherlands
• Alla Rozovskaya, Columbia University, USA
• Keisuku Sakaguchi, Johns Hopkins University, USA
• Mathias Schulze, University of Waterloo, Canada
• Serge Sharoff, University of Leeds, UK
• Swapna Somasundaran, ETS, USA
• Richard Sproat, Google, USA
• Helmer Strik, Radboug University Nijmegen, the Netherlands
• Nai-Lung Tsao, National Central University, Taiwan
• Lucy Vanderwende, Microsoft Research, USA
• Giulia Venturi, Institute of Computational Linguistics "Antonio Zampolli" (ILC-CNR), Italy
• Carl Vogel, Trinity College, Ireland
• Elena Volodina, University of Gothenburg, Sweden
• Monica Ward, Dublin City University, Ireland
• Pete Whitelock, Oxford University Press, UK
• Magdalena Wolska, University of Tubingen, Germany
• Peter Wood, University of Saskatchewan in Saskatoon, Canada
• Wenting Xiong, University of Pittsburgh, USA
• Huichao Xue, University of Pittsburgh, USA
• Helen Yannakoudakis, University of Cambridge, UK
• Marcos Zampieri, Saarland University, Germany
• Klaus Zechner, ETS, USA
• Torsten Zesch, University of Duisburg-Essen, Germany

Related Links

• ACL 2014
• 1st Workshop on Building Educational Applications Using NLP (2003)
• 2nd Workshop on Building Educational Applications Using NLP (2005)
• 3rd Workshop on Innovative Use of NLP for Building Educational Applications (2008)
• 4th Workshop on Innovative Use of NLP for Building Educational Applications (2009)
• 5th Workshop on Innovative Use of NLP for Building Educational Applications (2010)
• 6th Workshop on Innovative Use of NLP for Building Educational Applications (2011)
• 7th Workshop on Innovative Use of NLP for Building Educational Applications (2012)
• 8th Workshop on Innovative Use of NLP for Building Educational Applications (2013)

Other Upcoming Educational/NLP Events

• NIPS Workshop on Data-Driven Education (Lake Tahoe, USA; December 10, 2013)
• 3rd Workshop on Predicting and Improving Text Readability for Target Reader Populations (Gothenburg, Sweden; April 27, 2014)
• Traitement automatique des langues naturelles et apprentissage assisté par ordinateur (Montreal, Canada; May 15-16, 2014)
• Intelligent Tutoring Systems (Honolulu, Hawaii, USA; June 05-09, 2014)
• ACL Workshop on Argumentation Mining (Baltimore, USA; June 26, 2014)
• ACL Workshop on Speech and Language Processing for Assistive Technologies (SLPAT) (Baltimore, USA; June 26, 2014)
• CoNLL Shared Task on Grammatical Error Correction (Baltimore, USA; June 26-27, 2014)
• EUROCALL 2014 (Groningen, The Netherlands; August 20-23, 2014)
• SemEval Task 5: L2 Writing Assistant (Dublin, Ireland; August 23-30, 2014)
• COLING Tutorial on Grammatical Error Correction (Dublin, Ireland; August ??, 2014)
• Summer School on Learner Corpus Research: Corpus Approaches to Phraseology (Louvain, Belgium; September 08-12, 2014)
• Interspeech Special Session on Tutorial Dialogues and Spoken Dialogue Systems (Singapore; September 08-14, 2014)
• 1st Workshop on Natural Language Processing Techniques for Educational Applications (NLP-TEA-1) (Nara, Japan; November 30-04, 2014)
• CLP-2014 Shared Task on Grammatical Error Diagnosis for Learning Chinese as a Foreign Language (CFL) (Nara, Japan; November 30-04, 2014)