The 12th Workshop on Innovative Use of NLP for Building Educational Applications

Workshop Description

The BEA Workshop, one of the largest one-day workshops in the ACL community, is a leading venue for NLP innovation for educational applications. The workshop’s continuous growth illustrates an alignment between societal need and technological advances. NLP capabilities now support an array of learning domains, including writing, speaking, reading, science, and mathematics, as well as the related intra- (e.g., self-confidence) and inter-personal (e.g., collaboration) domains that support achievement in those domains. Automated writing evaluation (AWE) and speech scoring applications are commercially deployed in high-stakes assessment and instructional contexts, including primary and secondary educational settings. Commercially-deployed plagiarism detection is also prevalent. The current educational and assessment landscape in K-12, higher education, and adult learning (in academic and workplace settings) fosters a strong interest in technologies that yield analytics to support proficiency measures for complex constructs. For writing, there is a focus on innovation that supports writing tasks requiring source use, argumentative discourse, and factual content accuracy. For speech, there is an interest in advancing automated scoring to include the evaluation of discourse and content features in responses to spoken assessments. General advances in speech technology have promoted a renewed interest in spoken dialog and multimodal systems for instruction and assessment in, for example, workplace interviews and simulated teaching environments. The explosive growth of mobile applications for game-based and simulation applications for instruction and assessment is another place where NLP has begun to play a large role, especially in language learning.

NLP for educational applications has gained visibility outside of the NLP community. First, the Hewlett Foundation reached out to public and private sectors and sponsored two competitions – one for automated essay scoring, and another for scoring of short response items – to engage the larger scientific community in this enterprise. Subsequently, EdX, a non-profit enterprise founded by Harvard and MIT, announced the release of software designed to automatically grade text using AI technologies. Learning@Scale, a relatively new venue for NLP research in education, promotes interdisciplinary research on learning and teaching. Massive Open Online Courses (MOOCs) now incorporate AWE systems to manage the thousands of assignments that may be received during a single MOOC course. MOOCs for Refugees have more recently popped up in response to current social situations. Courses include language learning, and AWE and other NLP capabilities could support the coursework.

Research into education-related problems has also continued to grow within the NLP community. The last five years saw eight shared tasks (including five on grammatical error correction), two workshops on NLP and Computer-Assisted Language Learning (TEA series), and two sessions of the ACL devoted exclusively to the role of NLP in assisting language learners. Three of the shared tasks were sponsored by BEA. At ACL 2016, there were over 12 papers devoted to NLP in education, the highest number ever at a conference of this caliber. All of this has served to increase the visibility of, and interest in, the field of education applications in NLP.

The 12th workshop will have oral presentation sessions and a large poster session in order to maximize the amount of original work presented (the workshop normally has 50+ attendees). We will invite both full papers and short papers on topics including: automated scoring of textual and spoken responses, intelligent tutoring, peer review, grammatical error detection/correction, learner cognition, spoken dialog, multimodal applications, tools for teachers and test developers, and use of corpora. Research that incorporates NLP for use with mobile and game-based platforms will be of special interest. Specific topics include:

• Automated scoring/evaluation for written student responses
  • Content analysis for scoring/assessment
  • Analysis of the structure of argumentation
  • Grammatical error detection and correction
  • Discourse and stylistic analysis
  • Plagiarism detection
  • Machine translation for assessment, instruction and curriculum development
  • Detection of non-literal language (e.g., metaphor)
  • Sentiment analysis
  • Non-traditional genres (beyond essay scoring)
  
  
• Intelligent Tutoring (IT) and Game-based assessment that incorporates NLP
  • Game-based learning
  • Dialogue systems in education
  • Hypothesis formation and testing
  • Multi-modal communication between students and computers
  • Generation of tutorial responses
  • Knowledge representation in learning systems
  • Concept visualization in learning systems
  
  
• Learner cognition
  • Assessment of learners' language and cognitive skill levels
  • Systems that detect and adapt to learners' cognitive or emotional states
  • Tools for learners with special needs
  
  
• Use of corpora in educational tools
  • Data mining of learner and other corpora for tool building
  • Annotation standards and schemas / annotator agreement
  
  
• Tools and applications for classroom teachers and/or test developers
  • NLP tools for second and foreign language learners
  • Semantic-based access to instructional materials to identify appropriate texts
  • Tools that automatically generate test questions
  • Processing of and access to lecture materials across topics and genres
  • Adaptation of instructional text to individual learners' grade levels
  • Tools for text-based curriculum development
  • E-learning tools for personalized course content
  • Language-based educational games
  
  
• Descriptions and proposals for shared tasks
  
  
• Retrospective or survey papers on a particular NLP/Edu topic or field
  
  
• Vision papers about ideas discussing how the field should develop

NLI Shared Task 2017

The workshop will also host a Shared Task on Native Language Identification (NLI). NLI is the process of automatically identify the native language (L1) of a non-native speaker based solely on language that he or she produces in another language. Two previous shared tasks on NLI have been organized in which the task was to identify the native language of non-native speakers of English-based on essays and spoken responses they provided during a standardized assessment of academic English proficiency. The first shared task was based on the essays only and was also held with the BEA workshop in 2013. It was a total success with 29 teams competing, making it one of the largest shared tasks that year. Three years later, Computational Paralinguistics Challenge at Interspeech 2016 hosted a sub-challenge on identifying the native language based solely on the spoken responses.

This year's shared task combines the inputs from the two previous tasks. There will be three tracks: NLI on the essay only, NLI on the speech response only, and NLI using both responses from a test taker. We feel this will make for a more challenging shared task while building on the methods and results from the previous two shared tasks. The training and development data for the shared task will be available in February 2017.

The organizing committee for the shared task is: Aoife Cahill (Educational Testing Service), Keelan Evanini (Educational Testing Service), Shervin Malmasi (Harvard Medical School), Joel Tetreault (Grammarly).

More details can be found on the shared task website: https://sites.google.com/site/nlisharedtask/.

Submission Information

We will be using the EMNLP Submission Guidelines for the BEA12 Workshop this year. (Please note we initially had the ACL Guidelines as the point of reference, but use the EMNLP ones. The two are extremely similar and you shouldn't see any differences when it comes to formatting and paper length). Authors are invited to submit a full paper of up to 8 pages of content with unlimited pages for references. We also invite short papers of up to 4 pages of content, including unlimited pages for references. Final camera ready versions of accepted papers will be given an additional page of content to address reviewer comments.

Papers which describe systems are also invited to give a demo of their system. If you would like to present a demo in addition to presenting the paper, please make sure to select either "full paper + demo" or "short paper + demo" under "Submission Category" in the START submission page.

Previously published papers cannot be accepted. The submissions will be reviewed by the program committee. As reviewing will be blind, please ensure that papers are anonymous. Self-references that reveal the author's identity, e.g., "We previously showed (Smith, 1991) ...", should be avoided. Instead, use citations such as "Smith previously showed (Smith, 1991) ...".

We have also included conflict of interest in the submission form. You should mark all potential reviewers who have been authors on the paper, are from the same research group or institution, or who have seen versions of this paper or discussed it with you.

We will be using the START conference system to manage submissions: https://www.softconf.com/emnlp2017/bea2017/

As in the above, please make sure you are using the EMNLP Submission Guidelines. Thanks!

Important Dates

• Submission Deadline: Sunday, June 11 - 23:59 EST (New York City Time) [ Current EST ]
• Notification of Acceptance: Friday, June 30
• Camera-ready Papers Due: Friday, July 14
• Workshop: September 08

Presentation Information

Oral Presentations: Long papers accepted for oral presentations are allotted 20 minutes for the talk and 5 minutes for questions. Short papers that are accepted for oral presentations are allotted 15 minutes for the talk and 5 minutes for questions.

Poster Presentations: All papers accepted for a poster presentation will be presented in the session after lunch between 2:00 and 3:30. The posterboards will be self-standing, on top of tables (giving room for laptops, business cards, handouts, etc). The posters should be sized for A0 Landscape. Double-sided tape, pushpins, etc. for affixing the posters to the boards will be provided.

Workshop Program

Organizing Committee

• Joel Tetreault, Grammarly (primary contact)
• Jill Burstein, Educational Testing Services
• Ekaterina Kochmar, University of Cambridge
• Claudia Leacock, Consultant
• Helen Yannakoudakis, University of Cambridge

Program Committee

• David Alfter, University of Gothenburg, Sweden
• Dimitrios Alikaniotis, University of Cambridge, UK
• Rafael E. Banchs, Institute for Infocomm Research, Singapore
• Rajendra Banjade, University of Memphis, USA
• Timo Baumann, Carnegie Mellon University, USA
• Lee Becker, Hapara, USA
• Beata Beigman Klebanov, Educational Testing Service, USA
• Lisa Beinborn, Technische Universität Darmstadt, Germany
• Kay Berkling, Cooperative State University, Karlsruhe, Germany
• Delphine Bernhard, LiLPa, Université de Strasbourg, France
• Yevgeni Berzak, MIT, USA
• Sameer Bhatnagar, Polytechnique Montreal, Canada
• Serge Bibauw, KU Leuven & Université Catholique de Louvain, Belgium
• Kristy Boyer, University of Florida, USA
• Chris Brew, Digital Operatives LLC, USA
• Ted Briscoe, University of Cambridge, UK
• Chris Brockett, Microsoft Research, USA
• Julian Brooke, University of Melbourne, Australia
• Dominique Brunato, Istituto di Linguistica Computazionale (ILC-CNR), National Council of Research, Pisa (Italy), Italy
• Christopher Bryant, University of Cambridge, UK
• Andrew Caines, University of Cambridge, UK
• Xioabin Chen, Tübingen University, Germany
• Wei-Fan Chen, Academia Sinica, Taiwan
• Lei Chen, Educational Testing Service, USA
• Yeonsuk Cho, Educational Testing Service, USA
• Martin Chodorow, City University of New York, USA
• Shamil Chollampatt, National University of Singapore, Singapore
• Mark Core, University of Southern California, USA
• Scott Crossley, Georgia State University, USA
• Ronan Cummins, University of Cambridge, UK
• Luis Fernando D’Haro, Institute for Infocomm Research, A*STAR, Singapore
• Vidas Daudaravicius, VTEX Research, Lithuania
• Markus Dickinson, Indiana University, USA
• Yo Ehara, National Institute of Advanced Industrial Science and Technology, Japan
• Keelan Evanini, Educational Testing Service, USA
• Mariano Felice, University of Cambridge, UK
• Michael Flor, Educational Testing Service, USA
• Peter Foltz, Pearson, USA
• Hector-Hugo Franco-Penya, DIT, Ireland
• Thomas François, Université Catholique de Louvain , Belgium
• Michael Gamon, Microsoft Research, USA
• Dipesh Gautam, University of Memphis, Nepal
• Binyam Gebrekidan Gebre, Philips, Netherlands
• Kallirroi Georgila, University of Southern California, USA
• Cyril Goutte, National Research Council Canada, Canada
• Iryna Gurevych, TU Darmstadt, Germany
• Na-Rae Han, University of Pittsburgh, USA
• Jiangang Hao, Educational Testing Service, USA
• Rachel Harsley, University of Illinois at Chicago, USA
• Homa B. Hashemi, University of Pittsburgh, USA
• Trude Heift, Simon Fraser University, Canada
• Derrick Higgins, American Family Insurance, USA
• Andrea Horbach, University of Duisburg-Essen, Germany
• Chung-Chi Huang, Frostburg State University, USA
• Radu Tudor Ionescu, University of Bucharest, Romania
• Ross Israel, Factual, USA
• Pamela Jordan, University of Pittsburgh, USA
• Marcin Junczys-Dowmunt, Adam Mickiewicz University in Poznan, Poland
• Fazel Keshtkar, St. John's University Professor, USA
• Levi King, Indiana University, USA
• Sigrid Klerke, University of Copenhagen, Denmark
• Ekaterina Kochmar, University of Cambridge, UK
• Mamoru Komachi, Tokyo Metropolitan University, Japan
• Robert Krovetz, Lexical Research, USA
• Girish Kumar, Stanford University, Singapore
• Lun-Wei Ku, IIS, Academica Sinica, Taiwan
• Kristopher Kyle, University of Hawaii, USA
• Lung-Hao Lee, National Taiwan Normal University, Taiwan
• John Lee, City University of Hong Kong, Hong Kong
• James Lester, North Carolina State University, USA
• Chen Wee Leong, Educational Testing Service, USA
• Baoli Li, Henan University of Technology, China
• Diane Litman, University of Pittsburgh, USA
• Annie Louis, University of Essex, UK
• Anastassia Loukina, Educational Testing Service, USA
• Xiaofei Lu, Pennsylvania State University, USA
• Fabiana MacMillan, Rosetta Stone, USA
• Nitin Madnani, Educational Testing Service, USA
• Shervin Malmasi, Harvard Medical School, USA
• Liliana Mamani Sanchez, University College Dublin, Ireland
• Montse Maritxalar, University of the Basque Country, Spain
• Ditty Matthew, Indian Institute of Technology Madras, India
• Julie Medero, Harvey Mudd College, USA
• Beata Megyesi , Uppsala University, Sweden
• Mohsen Mesgar, Heidelberg Institute for Theoretical Studies, Germany
• Angeliki Metallinou, Amazon, USA
• Detmar Meurers, University of Tübingen, Germany
• Christian M. Meyer, Technische Universität Darmstadt, Germany
• Lisa Michaud, Aspect Software, USA
• Michael Mohler, Language Computer Corporation, USA
• Maria Moritz, Faculty of Mathematics and Computer Science, Germany
• Smaranda Muresan, Columbia University, USA
• William Murray, Pearson, USA
• Courtney Napoles, Johns Hopkins University, USA
• Hwee Tou Ng, National University of Singapore, Singapore
• Huy Nguyen, University of Pittsburgh, USA
• Nobal Niraula, Boeing Research & Technology, USA
• Simon Ostermann, Saarland University, Germany
• Alexis Palmer, University of North Texas, USA
• Ted Pedersen, University of Minnesota, Duluth, USA
• Isaac Persing, University of Texas at Dallas, USA
• Patti Price, PPRICE Speech and Language Technology Consulting, USA
• Martí Quixal, Universitat Oberta de Catalunya, Spain
• Zahra Rahimi, University of Pittsburgh, USA
• Lakshmi Ramachandran, A9.com, USA
• David Randolph, University of Illinois at Chicago, USA
• Sudha Rao, University of Maryland, USA
• Livy Real, IBM Research, Brazil
• Marek Rei, University of Cambridge, UK
• Robert Reynolds, UiT The Arctic University of Norway, Norway
• Brian Riordan, Educational Testing Service, USA
• Andrew Rosenberg, IBM, USA
• Alla Rozovskaya, Queens College (CUNY), USA
• Alexander Rush, Harvard, USA
• Anton Rytting, University of Maryland, USA
• Keisuke Sakaguchi, Johns Hopkins University, USA
• Elizabeth Salesky, MIT LL, USA
• Allen Schmaltz, Harvard University, USA
• Swapna Somasundaran, Educational Testing Service, USA
• Helmer Strik, Radboud University Nijmegen, Netherlands
• David Suendermann-Oeft, Educational Testing Service, USA
• Kaveh Taghipour, National University of Singapore, Singapore
• Fatemeh Torabi Asr, Indiana University, USA
• Yuen-Hsien Tseng, National Taiwan Normal University, Taiwan
• Sowmya Vajjala, Iowa State University, USA
• Giulia Venturi, Istituto di Linguistica Computazionale (ILC-CNR), Italy
• Aline Villavicencio, Federal University of Rio Grande do Sul, Brazil
• Carl Vogel, Computational Linguistics Lab, O'Reilly Institute, Ireland
• Elena Volodina, University of Gothenburg, Sweden
• Michael White , The Ohio State University, USA
• Denise Whitelock, The Open University, UK
• David Wible, National Central University, Taiwan
• Alistair Willis, Open University, UK
• Magdelena Wolska, Eberhard Karls Universität Tübingen, Germany
• Michael Wojatzki, University of Duisburg-Essen, Germany
• Huichao Xue, Google, USA
• Zheng Yuan, University of Cambridge, UK
• Marcos Zampieri, Saarland University, Germany
• Torsten Zesch, University of Duisburg-Essen, Germany
• Fan Zhang, University of Pittsburgh, USA
• Xiaodan Zhu, National Research Council Canada, Canada

Related Links

• 1st Workshop on Building Educational Applications Using NLP (2003)
• 2nd Workshop on Building Educational Applications Using NLP (2005)
• 3rd Workshop on Innovative Use of NLP for Building Educational Applications (2008)
• 4th Workshop on Innovative Use of NLP for Building Educational Applications (2009)
• 5th Workshop on Innovative Use of NLP for Building Educational Applications (2010)
• 6th Workshop on Innovative Use of NLP for Building Educational Applications (2011)
• 7th Workshop on Innovative Use of NLP for Building Educational Applications (2012)
• 8th Workshop on Innovative Use of NLP for Building Educational Applications (2013)
• 9th Workshop on Innovative Use of NLP for Building Educational Applications (2014)
• 10th Workshop on Innovative Use of NLP for Building Educational Applications (2015)
• 11th Workshop on Innovative Use of NLP for Building Educational Applications (2016)

Other 2016 and 2017 Educational/NLP Events

• AIED Special Issue on MARWIDE: M​ultidisciplinary A​pproaches to R​eading and W​riting I​ntegrated with D​isciplinary E​ducation (to be published Q1 2017)
• SLaTE (Stockholm, Sweden; August 25-26, 2017)
• Spoken CALL Shared Task (Stockholm, Sweden; August 25-26, 2017)
• 4th Learner Corpus Research Conference (Bolzano, Italy; October 05-07, 2017)