Study-Unit Description

Study-Unit Description


CODE SCE5201

 
TITLE Machine Learning and Pattern Recognition

 
UM LEVEL 05 - Postgraduate Modular Diploma or Degree Course

 
ECTS CREDITS 10

 
DEPARTMENT Systems and Control Engineering

 
DESCRIPTION Machine learning and pattern recognition algorithms and the underlying theory will be presented in this study-unit. Methods that learn models from training data for fixed basis, and subsequently for adaptive basis, regression and classification will be described. This leads to a description of kernel methods that make direct use of the training data during the prediction phase. Methods that make use of latent variables will then be described and shown how they can be used in data clustering.

The study-unit also describes in detail methods that can be used for the analysis of sequential and temporal data.

Study-Unit Aims:

The aims of this study-unit are to:

- describe the theory and use of fixed basis regression and classification methods;
- describe the theory and use of adaptive basis regression and classification methods and their implementation as neural networks;
- describe the theory and use of kernel and sparse kernel methods;
- introduce the concept of latent variables;
- describe the theory and use of Gaussian mixtures and the Expectation-Maximisation (EM) algorithm;
- describe the theory and use of the K-means clustering algorithm and its relation to the EM algorithm;
- describe the theory and use of Markov models and Linear Dynamic Systems for sequential and temporal data analysis.

Learning Outcomes:

1. Knowledge & Understanding:

By the end of the study-unit the student will be able to:

- represent a general linear combination with a chosen basis function;
- describe the probabilistic interpretation of fixed basis regression learning;
- explain the purpose of regularisation in regression modelling;
- explain the problem of overfitting and the choice of model complexity;
- explain the discriminant function given a linear model;
- derive the Fisher's linear discriminant and relate it to the least-squares discriminant;
- derive linear and quadratic discriminants using a probabilistic generative model;
- explain how a feed-forward network of functions can provide adaptive basis regression and classification;
- derive the method by which the weights of a multilayer perceptron may be determined;
- explain how regularisation may be used in neural networks;
- describe convolutional neural networks and explain their use in deep neural networks;
- explain the Nadarya-Watson model for Radial Basis Function networks;
- explain the concept underlying Support Vector Machines (SVM);
- explain the learning algorithm for SVMs;
- explain the concept of latent variables to define Mixtures of Gaussians;
- explain the Expectation-Maximisation (EM) algorithm;
- explain the K-means clustering algorithm and its relation to the EM algorithm;
- explain how Markov models can be used to represent sequential data;
- explain how a Hidden Markov Model can be used to model sequential data;
- explain how a Linear Dynamic System can be used to model sequential data.

2. Skills:

By the end of the study-unit:

- given a set of data, the student will be able to perform (fixed basis) regression and classification;
- given a set of data, the student will be able to design, train and test a multilayer perceptron for regression and classification;
- for a set of data, the student will be able to design, train and test a convolutional neural network;
- construct and test the validity of kernels;
- given a set of data, the student will be able to design, train and test a Radial Basis Function network;
- given a set of data, the student will be able to train a Support Vector Machine;
- given a set of data, the student will be able to use the Expectation-Maximisation algorithm to determine a Gaussian mixture model;
- given a set of data, the student will be able to use the K-means clustering algorithm to obtain data clusters;
- given sequential data with discrete latent variables, the student will be able to model this using a Hidden Markov Model;
- given sequential data with Gaussian latent variables, the student will be able to model this using a Linear Dynamical System.

Main Text/s and any supplementary readings:

Main Texts:

- Bishop C. M. (2006). Pattern Recognition and Machine Learning. New York: Springer Verlag.
- Haykin S. (2007). Neural Networks - A Comprehensive Foundation. 3rd ed. New Jersey: Prentice-Hall, Inc.

Supplementary Readings:

- Duda R. O., Hart P. E. and Stork D. G. (2001). Pattern Classification. New York: John Wiley & Sons.

 
STUDY-UNIT TYPE Ind Study, Lecture, Practicum, Project & Tutorial

 
METHOD OF ASSESSMENT
Assessment Component/s Assessment Due Sept. Asst Session Weighting
Practical SEM2 No 30%
Project SEM2 No 30%
Examination (2 Hours) SEM2 Yes 40%

 
LECTURER/S Kenneth P. Camilleri

 

 
The University makes every effort to ensure that the published Courses Plans, Programmes of Study and Study-Unit information are complete and up-to-date at the time of publication. The University reserves the right to make changes in case errors are detected after publication.
The availability of optional units may be subject to timetabling constraints.
Units not attracting a sufficient number of registrations may be withdrawn without notice.
It should be noted that all the information in the description above applies to study-units available during the academic year 2023/4. It may be subject to change in subsequent years.

Study at UM

https://www.um.edu.mt/course/studyunit