| CODE | CCE5110 | |||||||||
| TITLE | Best Practice in Model Development and Training | |||||||||
| UM LEVEL | 05 - Postgraduate Modular Diploma or Degree Course | |||||||||
| MQF LEVEL | 7 | |||||||||
| ECTS CREDITS | 5 | |||||||||
| DEPARTMENT | Communications and Computer Engineering | |||||||||
| DESCRIPTION | The study-unit complements theoretical courses in machine learning and covers processes typically followed in building robust models ready for production, from data collection and organisation, infrastructure for training and testing models, evaluation of such models and conducting error analysis, amongst others. The study-unit complements theoretical courses in machine learning and covers processes followed in building robust models ready for production. The following topics are covered: Data collection, curation and organisation; dealing with class imbalance, data augmentation and feature engineering; Planning infrastructure for the training and testing of the models; Tools for the evaluation of models; conducting error analysis; dealing with data distribution shifts; continuous learning and testing in production; Human team composition and structure. Study-Unit Aims: The aim of the study-unit is to prepare the student for the professional world in a machine learning engineering setup. The study-unit aims to cover best practices throughout the life-cycle of signal processing and machine learning models, covering data preparation, training cycle, evaluation, error analysis and dealing with data drift user experience and the various human roles in the development team. Learning Outcomes: 1. Knowledge & Understanding: By the end of the study-unit the student will be able to: - Articulate and critique data collection methodologies, including their appropriateness, reliability, and ethical considerations within various contexts; - Analyze and optimize dataset preparation workflows, encompassing data cleaning, transformation, augmentation, and integration strategies to ensure data readiness for modeling; - Compare and evaluate diverse problem representation techniques, including structured vs unstructured data, and approaches suited for classification, regression, and clustering problems; - Critically examine model evaluation strategies, including cross-validation, stratification, and the use of performance metrics like AUC-ROC, precision-recall, and F1-score for balanced and imbalanced datasets; - Explore and implement regularization techniques and hyperparameter tuning algorithms such as grid search, random search, Bayesian optimization, and automated machine learning; - Identify and apply robust feature selection methods, including filter, wrapper, and embedded techniques, to improve model performance and interpretability; - Diagnose and interpret model performance through rigorous error analysis techniques, identifying sources of bias, variance, and anomalies; - Design and conduct ongoing model monitoring, incorporating concept drift detection and re-evaluation mechanisms to maintain long-term model accuracy and relevance. 2. Skills: By the end of the study-unit the student will be able to: - Select an appropriate dataset sampling and splitting technique; - Select an appropriate model evaluation technique; - Implement the model training pipeline including hyper parameter optimisation; - Monitor the training process and adjust as necessary; - Carry out error analysis and suggest changes to the data or model. Main Text/s and any supplementary readings: Main Texts: - Valliappa Lakshmanan, Sara Robinson, Michael Munn , “Machine Learning Design Patterns” (2020) O'Reilly Media, Inc. ISBN: 9781098115784. Supplementary Readings: - Lu, Jie, et al. “Learning under concept drift: A review.” IEEE transactions on knowledge and data engineering 31.12 (2018): 2346–2363. - Sculley, David, et al. “Hidden technical debt in machine learning systems.” Advances in neural information processing systems 28 (2015). |
|||||||||
| ADDITIONAL NOTES | Pre-requisite Qualifications: Computer Programming Co-requisite Study-units: Machine Learning |
|||||||||
| STUDY-UNIT TYPE | Lecture | |||||||||
| METHOD OF ASSESSMENT |
|
|||||||||
| LECTURER/S | ||||||||||
|
The University makes every effort to ensure that the published Courses Plans, Programmes of Study and Study-Unit information are complete and up-to-date at the time of publication. The University reserves the right to make changes in case errors are detected after publication.
The availability of optional units may be subject to timetabling constraints. Units not attracting a sufficient number of registrations may be withdrawn without notice. It should be noted that all the information in the description above applies to study-units available during the academic year 2025/6. It may be subject to change in subsequent years. |
||||||||||