| CODE | LLT3510 | ||||||
| TITLE | Deep Learning Approaches to Natural Language Processing | ||||||
| UM LEVEL | 03 - Years 2, 3, 4 in Modular Undergraduate Course | ||||||
| MQF LEVEL | 6 | ||||||
| ECTS CREDITS | 6 | ||||||
| DEPARTMENT | Institute of Linguistics and Language Technology | ||||||
| DESCRIPTION | This study-unit focuses on techniques for solving Natural Language Processing tasks through the design of neural architectures. The contents of the study-unit can be divided into two main components: Methods - Overview of Machine Learning fundamentals, including experimental design, evaluation metrics and data processing issues; - The relationship and differences between linear, log-linear and neural models; - Feed-forward neural models for NLP; - Convolutional neural network models for NLP; - Recurrent networks (including Long Short-Term Memory and Gated Recurrent Unit models); - Encoder-decoder architectures and attention; - Attention-based (Transformer) networks. NLP Challenges The above core methodological topics are in turn applied to a variety of natural language analysis and generation tasks, including, but not limited to: - Language models; - Lexical semantics and vector-space semantic models; - Text classification, including sentiment analysis, emotion classification and topic classification; - Sequence classification such as morphosyntactic labelling and named entity recognition; - Deep semantics, with a special focus on Natural Language Inference and Textual Similarity; - Conditioned Generation, including data-to-text generation and machine translation. The topics will be interleaved, such that a given methodological topic is exemplified with reference to a core NLP challenge. The study-unit will be structured along weekly lectures, with bi-weekly practical sessions which will introduce the most important frameworks for neural modelling (such as Tensorflow and/or PyTorch) and more broadly, the Python ecosystem for handling data and numerical computing (pandas, numpy etc). Practicals will take the form of tasks in which data will be provided, corresponding to the challenges and topics addressed in class. Study-Unit Aims: Contemporary NLP applications increasingly rely on the design of architectures with neural components, which learn from large repositories of linguistic data in a supervised or self-supervised way. In view of this, this study-unit aims to: - give students a thorough grounding in the formal and conceptual foundations of neural methods; - show how these methods are deployed in the construction of systems for the robust analysis and generation of Natural Language; - pave the way for other units in speech processing, information extraction, and multilingual computing. Learning Outcomes: 1. Knowledge & Understanding: By the end of the study-unit the student will be able to: - identify the appropriate neural architecture to address a particular NLP problem; - distinguish the strengths and weaknesses of recurrent models (including LSTM/GRU models) and attention-based models; - design architectures composed of different components for training in an end-to-end fashion for Natural Language Understanding and Natural Language Generation tasks; - implement such architectures in a high-level programming language; - evaluate outcomes using appropriate evaluation metrics, notably accuracy, precision, recall and F1 measures, as well as string-based metrics for generation tasks. 2. Skills: By the end of the study-unit the student will be able to: - formulate a research problem in probabilistic terms; - formulate and test hypotheses; - manage large datasets efficiently, and handle sampling issues to create training/testing/development splits; - implement a machine-learning experiment to address a specific NLP problem in a high-level programming language; - use Python libraries to implement deep learning models to solve classification problems in NLP. Main Text/s and any supplementary readings: Main Texts: - D. Jurafsky and H Martin (2009). Speech and language processing (2nd Ed). New York: Prentice Hall [2nd Edition available in the library. Third edition available online at https://web.stanford.edu/~jurafsky/slp3/). Students should always consult the third edition unless otherwise specified.] - Y. Goldberg (2017). Neural network methods for Natural Language Processing. Morgan & Claypool Publishers. Supplementary Readings: - I. Goodfellow, Y. Bengio and A. Courville. (2016). Deep learning. Cambridge, MA: MIT Press. See also the companion website: https://www.deeplearningbook.org - A. Geron (2017). Hands-on machine learning with Scikit-Learn and Tensorflow. Sebastopol, CA: O'Reilly. |
||||||
| ADDITIONAL NOTES | Pre-requisite Qualifications: Sound knowledge of statistics. Prior exposure to machine learning methods. Basic knowledge of linguistics. | ||||||
| STUDY-UNIT TYPE | Lecture and Practicum | ||||||
| METHOD OF ASSESSMENT |
|
||||||
| LECTURER/S | Marc Tanti |
||||||
|
The University makes every effort to ensure that the published Courses Plans, Programmes of Study and Study-Unit information are complete and up-to-date at the time of publication. The University reserves the right to make changes in case errors are detected after publication.
The availability of optional units may be subject to timetabling constraints. Units not attracting a sufficient number of registrations may be withdrawn without notice. It should be noted that all the information in the description above applies to study-units available during the academic year 2023/4. It may be subject to change in subsequent years. |
|||||||