The 10th Workshop on Innovative Use of NLP for Building Educational Applications

Pre-Workshop Information

For those attending the BEA10 Workshop this Thursday, June 04, there are a few notes:

• Proceedings: The proceedings for the BEA10 can be found here.

• Poster Session: This year we are having two back-to-back poster sessions which will allow people to better circulate and see more posters. Each session is 45 minutes and consists of 10 papers each. Please see the workshop program below to see which poster is in which session.

• Oral Presentations: For those presenting orally, please send your presentation to bea.nlp.workshop@gmail.com in advance or come early to the workshop between 8:45 - 9:00 AM so we can load the presentations on one machine to make the transitions between talks more seamless.

• BEA10 T-shirts: This year is the third year we've had sponsors and we are using a portion of the sponsorship to fund our first ever workshop t-shirts. T-shirts are available for all who register. We will be distributing the t-shirts at the workshop on Thursday. Please make sure you have proof of registration with you. We ordered a range of sizes but if you want the best chance of getting a size that fits you, please come to the workshop early on Thursday. There is one t-shirt per registrant.

• Post-Workshop Dinner: As in previous editions, we will once again be hosting our "world famous" post-workshop dinner. This year's will be at 7:00 at Rio Grande, located at 1525 Blake Street, a nice 20 minute walk from the conference hotel along 16th Street. Directions can be found here. In the morning of the workshop, we will be getting a head count to know how many seats to reserve, so please let us know then or by email beforehand. Finally, we will be using the remaining sponsorship funds to subsidize the dinner for students.

Post-Workshop Information

• Links to papers can be found on the ACL Anthology, as well as below in the workshop program.
• Slides for oral presentations can also be found in the workshop program.
• If you're interested in joining the BEA Email List to get bi-monthly updates on the latest in education and NLP, please fill out this form.

Workshop Description

We are excited to be proposing a 10th anniversary BEA workshop. Since starting in 1997, the BEA workshop, now one of the largest workshops at NAACL/ACL, has become one of the leading venues for publishing innovative work which uses NLP to develop educational applications.

The consistent interest and growth of the workshop has clear ties to societal need and related advances in the technology, and the maturity of the NLP/education field. NLP capabilities now support an array of learning domains, including writing, speaking, reading, and mathematics. Within these domains, the community continues to develop and deploy innovative NLP approaches for use in educational settings. In the writing and speech domains, automated writing evaluation (AWE) and speech scoring applications, respectively, are commercially deployed in high-stakes assessment and instructional settings, including Massive Open Online Courses (MOOCs). We also see widely-used commercial applications for plagiarism detection and peer review. Major advances in speech technology, have made it possible to include speech in both assessment and Intelligent Tutoring Systems (ITS). There has been a renewed interest in spoken dialog and multi-modal systems for instruction and assessment. We are also seeing explosive growth of mobile applications for game-based applications for instruction and assessment. The current educational and assessment landscape, especially in the United States, continues to foster a strong interest and high demand that pushes the state-of-the-art in AWE capabilities to expand the analysis of written responses to writing genres other than those traditionally found in standardized assessments, especially writing tasks requiring use of sources and argumentative discourse.

The use of NLP in educational applications has gained visibility outside of the NLP community. First, the Hewlett Foundation reached out to public and private sectors and sponsored two competitions: one for automated essay scoring , and the other for scoring of short answer, fact-based response items . The motivation driving these competitions was to engage the larger scientific community in this enterprise. MOOCs are now beginning to incorporate AWE systems to manage the thousands of constructed-response assignments collected during a single MOOC course. Learning@Scale is a new venue for discussing NLP research in education. Another breakthrough for educational applications within the CL community is the presence of a number of shared-task competitions over the last three years. There have been three shared tasks on grammatical error correction with the most recent edition hosted at CoNLL 2014. In 2014 alone, there were four shared tasks for NLP and Education-related areas.

In 2015, we expect that the workshop (consistent with the nine previous workshops at ACL and NAACL/HLT), will continue to expose the NLP research community to technologies that identify novel opportunities for the use of NLP techniques and tools in educational applications. This BEA10 workshop will solicit both full papers and short papers for oral and poster presentations. We will solicit papers for educational applications that incorporate NLP methods, including, but not limited to: automated scoring of open-ended textual and spoken responses; game-based instruction and assessment; intelligent tutoring; peer review, grammatical error detection; learner cognition; spoken dialog; multi-modal applications; tools for teachers and test developers; and use of corpora. Research that incorporates NLP methods for use with mobile and game-based platforms, and academic ePortfolio systems or MOOCs continues to be of special interest. Finally, as this is the 10th anniversary, we invite papers which provide a retrospective view, reflecting on past and current trends in the field, and vision papers which illustrate research directions for growth in the field. Specific topics include:

• Automated scoring/evaluation for written student responses
  • Content analysis for scoring/assessment
  • Analysis of the structure of argumentation
  • Grammatical error detection and correction
  • Discourse and stylistic analysis
  • Plagiarism detection
  • Machine translation for assessment, instruction and curriculum development
  • Detection of non-literal language (e.g., metaphor)
  • Sentiment analysis
  • Non-traditional genres (beyond essay scoring)
  
  
• Intelligent Tutoring (IT) and Game-based assessment that incorporates NLP
  • Dialogue systems in education
  • Hypothesis formation and testing
  • Multi-modal communication between students and computers
  • Generation of tutorial responses
  • Knowledge representation in learning systems
  • Concept visualization in learning systems
  
  
• Learner cognition
  • Assessment of learners' language and cognitive skill levels
  • Systems that detect and adapt to learners' cognitive or emotional states
  • Tools for learners with special needs
  
  
• Use of corpora in educational tools
  • Data mining of learner and other corpora for tool building
  • Annotation standards and schemas / annotator agreement
  
  
• Tools and applications for classroom teachers and/or test developers
  • NLP tools for second and foreign language learners
  • Semantic-based access to instructional materials to identify appropriate texts
  • Tools that automatically generate test questions
  • Processing of and access to lecture materials across topics and genres
  • Adaptation of instructional text to individual learners' grade levels
  • Tools for text-based curriculum development
  • E-learning tools for personalized course content
  • Language-based educational games
  
  
• Descriptions and proposals for shared tasks
  
  
• Retrospective or survey papers on a particular NLP/Edu topic or field
  
  
• Vision papers about ideas discussing how the field should develop

Submission Information

We will be using the NAACL 2015 Submission Guidelines for the BEA10 Workshop this year. Authors are invited to submit a full paper of up to 9 pages of content with up to 2 additional pages for references. We also invite short papers of up to 5 pages of content, including 2 additional pages for references. Please note that unlike previous years, final, camera ready versions of accepted papers will not be given an additional page to address reviewer comments.

Papers which describe systems are also invited to give a demo of their system. If you would like to present a demo in addition to presenting the paper, please make sure to select either "full paper + demo" or "short paper + demo" under "Submission Category" in the START submission page.

Previously published papers cannot be accepted. The submissions will be reviewed by the program committee. As reviewing will be blind, please ensure that papers are anonymous. Self-references that reveal the author's identity, e.g., "We previously showed (Smith, 1991) ...", should be avoided. Instead, use citations such as "Smith previously showed (Smith, 1991) ...".

Please use the 2015 NAACL style sheets for composing your paper: http://naacl.org/naacl-pubs/ .

We will be using the START conference system to manage submissions: https://www.softconf.com/naacl2015/bea

Important Dates

• Submission Deadline: March 08 - 23:59 EST (New York City Time) [ Current EST ]
• Notification of Acceptance: March 24
• Camera-ready Papers Due: April 03
• Workshop: June 04

Presentation Information

Oral Presentations: Long papers accepted for oral presentations are allotted 20 minutes for the talk and 5 minutes for questions. Short papers that are accepted for oral presentations are allotted 15 minutes for the talk and 5 minutes for questions.

Poster Presentations: All papers accepted for a poster presentation will be presented in the session after lunch between 2:00 and 3:30. The posterboards will be self-standing, on top of tables (giving room for laptops, business cards, handouts, etc). The posterboards measure 36 inches high and 48 inches wide . Double-sided tape, pushpins, etc. for affixing the posters to the boards will be provided.

Workshop Program

Program Committee

• Lars Ahrenberg, Linköping University, Sweden
• Laura Allen, Arizona State University, USA
• Timo Baumann, Universität Hamburg, Germany
• Lee Becker, Hapara, USA
• Beata Beigman Klebanov, Educational Testing Service, USA
• Delphine Bernhard, LiLPa, Université de Strasbourg, France
• Suma Bhat, University of Illinois, USA
• Kristy Boyer, North Carolina State University, USA
• Chris Brew, Thomson-Reuters Research, UK
• Ted Briscoe, University of Cambridge, UK
• Chris Brockett, Microsoft Research, USA
• Julian Brooke, University of Toronto, Canada
• Aoife Cahill, Educational Testing Service, USA
• Min Chi, North Carolina State University, USA
• Martin Chodorow, Educational Testing Service & CUNY, USA
• Mark Core, University of Southern California, USA
• Markus Dickinson, Indiana University, USA
• Myroslava Dzikovska, University of Edinburgh, UK
• Keelan Evanini, Educational Testing Service, USA
• Mariano Felice, University of Cambridge, UK
• Michael Flor, Educational Testing Service, USA
• Jennifer Foster, Dublin City University, Ireland
• Thomas François, Université Catholique de Louvain, Belgium
• Anette Frank, Heidelberg University, Germany
• Michael Gamon, Microsoft Research, USA
• Binyam Gebrekidan Gebre, Max Planck Institute for Psycholinguistics, Netherlands
• Kallirroi Georgila, University of Southern California, USA
• Dan Goldwasser, Purdue University, USA
• Cyril Goutte, National Research Council, Canada
• Iryna Gurevych, University of Darmstadt, Germany
• Trude Heift, Simon Fraser University, Canada
• Michael Heilman, Civis Analytics, USA
• Derrick Higgins, Civis Analytics, USA
• Andrea Horbach, Saarland University, Germany
• Chung-Chi Huang, National Institutes of Health, USA
• Radu Ionescu, University of Bucharest, Romania
• Ross Israel, Factual, USA
• Levi King, Indiana University, USA
• Ola Knutsson, Stockholm University, Sweden
• Ekaterina Kochmar, University of Cambridge, UK
• Mamoru Komachi, Tokyo Metropolitan University, Japan
• Lun-Wei Ku, Academia Sinica, Taiwan
• John Lee, City University of Hong Kong, Hong Kong
• Sungjin Lee, Yahoo Labs, USA
• Samuel Leeman-Munk, North Carolina State University, USA
• Chee Wee (Ben) Leong, Educational Testing Service, USA
• James Lester, North Carolina State University, USA
• Annie Louis, University of Edinburgh, UK
• Anastassia Loukina, Educational Testing Service, USA
• Xiaofei Lu, Penn State University, USA
• Wencan Luo, University of Pittsburgh, USA
• Nitin Madnani, Educational Testing Service, USA
• Shervin Malmasi, Macquarie University, Australia
• Montse Maritxalar, University of the Basque Country, Spain
• Mourad Mars, Umm Al-Qura University, KSA
• Aurélien Max, LIMSI-CNRS \& Univ. Paris Sud, France
• Julie Medero, Harvey Mudd College, USA
• Detmar Meurers, Universität Tübingen, Germany
• Lisa Michaud, Merrimack College, USA
• Rada Mihalcea, University of Michigan, USA
• Michael Mohler, Language Computer Corporation, USA
• Jack Mostow, Carnegie Mellon University, USA
• Smaranda Muresan, Columbia University, USA
• Ani Nenkova, University of Pennsylvania, USA
• Hwee Tou Ng, National University of Singapore, Singapore
• Rodney Nielsen, University of North Texas, USA
• Alexis Palmer, IMS at University of Stuttgart, Germany
• Aasish Pappu, Yahoo Labs, USA
• Ted Pedersen, University of Minnesota, Duluth, USA
• Ildiko Pilan, University of Gothenburg, Sweden
• Heather Pon-Barry, Mount Holyoke College, USA
• Patti Price, PPRICE Speech and Language Technology, USA
• Martí Quixal Martinez, Eberhard Karls Universität Tübingen, Germany
• Lakshmi Ramachandran, Pearson, USA
• Vikram Ramanarayanan, Educational Testing Service, USA
• Arti Ramesh, University of Maryland, College Park, USA
• Andrew Rosenberg, CUNY Queens College, USA
• Mihai Rotaru, Textkernel, Netherlands
• Alla Rozovskaya, Columbia University, USA
• C. Anton Rytting, University of Maryland, USA
• Keisuke Sakaguchi, Johns Hopkins University, USA
• Elizabeth Salesky, MITLL, USA
• Mathias Schulze, University of Waterloo, USA
• Serge Sharoff, University of Leeds, UK
• Swapna Somasundaran, Educational Testing Service, USA
• Richard Sproat, Google, USA <
• Helmer Strik, Radboud University Nijmegen, Netherlands
• David Suendermann-Oeft, Educational Testing Service, USA
• Sowmya Vajjala, Universität Tübingen, Germany
• Carl Vogel, Trinity College, Ireland
• Elena Volodina, University of Gothenburg, Sweden
• Xinhao Wang, Educational Testing Service, USA
• Denise Whitelock, The Open University, UK
• Magdalena Wolska, Eberhard Karls Universität Tübingen, Germany
• Peter Wood, University of Saskatchewan, Canada
• Huichao Xue, University of Pittsburgh, USA
• Marcos Zampieri, Saarland University, Germany
• Klaus Zechner, Educational Testing Service, USA
• Torsten Zesch, University of Duisburg-Essen, Germany
• Fan Zhang, University of Pittsburgh, USA
• Xiaodan Zhu, National Research Council, Canada

Related Links

• NAACL 2015
• 1st Workshop on Building Educational Applications Using NLP (2003)
• 2nd Workshop on Building Educational Applications Using NLP (2005)
• 3rd Workshop on Innovative Use of NLP for Building Educational Applications (2008)
• 4th Workshop on Innovative Use of NLP for Building Educational Applications (2009)
• 5th Workshop on Innovative Use of NLP for Building Educational Applications (2010)
• 6th Workshop on Innovative Use of NLP for Building Educational Applications (2011)
• 7th Workshop on Innovative Use of NLP for Building Educational Applications (2012)
• 8th Workshop on Innovative Use of NLP for Building Educational Applications (2013)
• 9th Workshop on Innovative Use of NLP for Building Educational Applications (2014)

Other Upcoming Educational/NLP Events

• 4th Workshop on NLP for Computer-Assisted Language Learning (NLP4CALL) (Vilnius, Lithuania; May 11, 2015)
• 2nd Workshop on Natural Language Processing Techniques for Educational Applications (*NLP-TEA-2*) (Beijing, China; July 31, 2015)
• Chinese Grammatical Error Diagnosis Shared Task (Beijing, China; July 31, 2015)
• SLaTE Workshop on L1 Teaching, Learning and Technology (Leipzig, Germany; September 03, 2015)
• 13th Annual Conference on Technology for Second Language Learning (TSLL 2015) "Data-driven Approaches to Learning Phraseology and Formulaic Language: Computation, Co-selection, Contextualization, Cognition" (Ames, IA, USA; September 18-19, 2015)
• ICDM Wksh on Data Mining for Educational Assessment and Feedback (ASSESS 2015) Atlantic City, NJ, USA; November 14, 2015)