OAR@UM Collection:

A comparison of penalized regression techniques

Wed, 01 Jan 2014 00:00:00 GMT

Title: A comparison of penalized regression techniques Abstract: The Ordinary Least Squares method (OLS) as defined by Carl Fredrich Gauss in the 18th century is a technique which is widely used to estimate parameter coefficients. However throughout the years as researchers were studying the stability of such technique it was noted that if the data is characterized by multicollinearity then the coefficient estimates obtained through the OLS proved to be weak. Thus, researchers after recognizing such weakness within the OLS framework embarked on a journey to develop new regression techniques which can provide stable and reliable results given that the data exhibits collinearity problems. These regression techniques include the Ridge regression, Least Absolute Shrinkage Selection Operator (Lasso) regression, Elastic net and Naive Elastic net regressions. The Ridge regression shall be introduced briefly however the main focus of this dissertation will be on the latter three techniques. The Lasso regression has the ability to minimize the effect of multicollinearity by applying shrinkage on the coefficient estimates while at the same time doing subset selection. Thus, unlike the model fitted by the Ridge regression, the resultant model of the Lasso is a parsimonious one. The Elastic Net regression (EN) and Naive Elastic Net (NEN) regression can be considered as hybrid models of the Ridge regression and Lasso regression whereby through their penalty function they also minimize the effect of shrinkage and also apply subset selection. The novelty of the EN and NEN regression lies with their ability to tackle the problem that arises in the Lasso regression whereby if a group of highly correlated variables exist in the dataset, the Lasso tends to choose a variable randomly from within this group. This is generally known as the "grouping effect" in literature. These techniques shall be implemented into two types of studies, a simulation study and a real life dataset study in order to analyse and compare their performance under different scenarios. In the simulation study, three datasets with different levels of multicollinearity and dimensions shall be analyzed while for the real life dataset, the high dimensional case ( n < p) shall be studied. Description: B.SC.(HONS)STATS.&OP.RESEARCH

Modeling ordinal data through bayesian sem

Sun, 01 Jan 2012 00:00:00 GMT

Title: Modeling ordinal data through bayesian sem Abstract: Structural equation modeling (SEM) is a flexible statistical technique used to model complex relationships between a set of observable and unobservable variables, where each variable may be either dependent or independent. A SEM consists of a measurement model, catering for the relationships between the observable and latent variables, and a structural model, a simultaneous equation indicating how the latent variables in the model are related to each other. Once an appropriately identified SEM has been specified, the step to follow is that of estimating the unknown parameters in the model. As an estimation technique, the Bayesian approach has become increasingly popular in the field of SEM particularly when dealing with small samples. On using this approach, the unknown model parameters are estimated through the technique of data augmentation and the Markov Chain Monte Carlo (MCMC) methods. Once estimates are obtained, the goodness-of-fit of the model is assessed through the posterior predictive p-value, a Bayesian alternative to the classical p-value. The Bayesian SEM strategy will be applied to examine the relationship between a set of observable and latent variables in a dataset related to invasion of privacy, risk taking and security concerns of a person when using the internet. Description: B.SC.(HONS)STATS.&OP.RESEARCH

Evolutionary algorithms

Tue, 01 Jan 2013 00:00:00 GMT

Title: Evolutionary algorithms Abstract: Evolutionary Algorithms are probabilistic techniques that are inspired by the principle of natural evolution proposed by Charles Darwin. Evolutionary Algorithms are normally used to generate useful solutions to optimization and search problems using methods that are borrowed from the principles of natural evolution such as selection, crossover and mutation. An overview of these algorithms especially a particular class of these algorithms named Genetic Algorithm is provided. A description of the function of Evolutionary Algorithms together with an outline of the main classes into which it is divided is given. Consequently, a description of each component in the algorithm is presented. Moreover, Evolutionary Algorithms are modelled by Markov Processes thus the Markov model of the algorithm together with the conditions under which the algorithm with an elitist selection rule converges to the global minimum of an optimization problem irrespective of the search space is provided. Genetic Algorithm is one of the main classes of Evolutionary Algorithms. A description of the main components of this algorithm together with some examples on how these can be incorporated is given. A Genetic Algorithm is also modelled by Markov processes and the exact transition matrix of a Genetic Algorithm is presented. Furthermore, fifteen benchmark functions are used to test the efficiency and the performance of the Genetic Algorithm. The parameters considered for this analysis include the population size and the crossover rate. Also, Genetic Algorithm is compared to other traditional methods, namely Nelder and Mead Moving Simplex, Simulated Annealing and Pattern Search to find in which cases Genetic Algorithm performs better than the other techniques. Description: B.SC.(HONS)STATS.&OP.RESEARCH

Modelling online credit card usage

Thu, 01 Jan 2009 00:00:00 GMT

Title: Modelling online credit card usage Abstract: The aim of this dissertation is to model the spending behavior of credit cardholders, with the idea of classifying legitimate and fraudulent users separately. The utilisation of Markov theory, in particular the Hidden Markov Model (HMM), was a valuable tool in providing solutions to the three main problems tackled. Dynamic Programming methods were essential in order to reduce the demanding computation of several calculations. Particularly, the Viterbi algorithm was exploited in the second HMM problem known as the Decoding Problem. Theory from Maximum Likelihood was also used especially in the third HMM problem, usually termed as the Learning Problem. A famous Expectation-Maximisation (EM) algorithm called the Baum-Welch algorithm was used so as to obtain the most likely parameters that best describe each credit cardholder's spending patterns. Finally, classification of legitimate and fraudulent users was carried out using cluster analysis. Description: B.SC.(HONS)STATS.&OP.RESEARCH