Representation does matter

Abstract

In Machine Learning the main problem is that of learning a ‘description’ of a class (possibly an infinite set) from a finite num- ber of positive and negative training examples. For real world problems, however, one must distinguish between the actual instance of the class to be learned and the numeric or symbolic encoding of the instances of the same class. The question here is whether different encodings (or representations) of the instances of a real-world class can actually affect the performance of the learning algorithm. In artificial neural networks (ANNs), for example, it is required that the classes are always encoded as vectors over some field (usually the set of reals). In this paper it is argued that the representation of the class instances plays a very important role in machine learning since it has bearing on two very important issues — the structural completeness of the training set and also the inductive bias of the learning algorithm.