A common problem in speech processing and recognition is the presence
of noise in the signal. Conventional techniques use some or other
creative and, often ingenious and/or complex, way to estimate and model
the noise source, so as to be able to filter it out afterwards.
We thought it might be a fun idea try a different approach. Rather
than modelling the noise, it might be worthwhile modelling the speech
source (i.e. you) itself. In this way, the relevant part of the signal
(the part that goes blahblahblah, rather than shshshshshshshsh) can be
extracted from the noisy signal. The problem is of course how to model
speech. We want to try and adopt a very simple approach by building a
model based on the quantisation of the spectral info of speech. We, or
rather you, will do this using self-organising neural networks and some
creative solutions to overcome the problems that come free of charge
using any strategy.
Thesis done by Pieter-Jan Ghesqière
Thesis done by Herwig De Smet
The purpose of topographic map formation is to dynamically adapt the neurons of the map so that nearboring positions in the input space are encoded by neighboring neurons. In contrast with the most widely used original learning paradigm developed for this matter, its constructive variants, are believed to better capture the fine-structure of the input density distribution since these variants incorporate a dynamic, self-organizing behavior. For example, the growing cell structures algorithm (Fritzke, 1992) uses a fixed, simplex-based neighborhood topology; ew units are inserted close to units which have a too high activation probability; existing units are deleted if their activation probabilities are too low (i.e. a sort of "conscience"). The neural gas network (Martinetz and Schulten, 1991) also generates topology-preserving mappings but modifies the topology locally by inserting/deleting edges depending on their "age." The incremental radial basis function (RBF) approach of Fritzke (1994) is still different since it starts from a small, "maximally"-ordered map and gradually inserts new units according to the same "age"-related metric as for the neural gas network. Recently, we have develop a stable constructive algorithm, called the stable growing gas algorithm. The purpose of this thesis is to compare the performance of these algorithms, by devising key test examples for it, and to test their ability to perform fast learning of rare events (one-shot learning).
Thesis done by Lu Yang
Kubovy & Wagemans have quantified and tested the old Gestalt law of grouping by proximity (Kubovy & Wagemans, 1995, Psych. Science). They have also demonstrated experimentally that grouping operates locally, in a way that is unaffected by the configuration within which it is embedded, i.e., against the famous Gestalt motto that the whole is more than the sum of its parts (Kubovy, Holcombe & Wagemans, 1998, Cogn. Psych.). We now want to develop a more mechanistic account of these grouping principles in a way that does justice to current knowledge of the neuroanatomy and -physiology of the visual system (e.g., orientation-tuned cells in V1). The goal of this thesis was building a neural network implementing its major principles. This should allow further tests of the feasibility and psychological plausibility of this account. This topic was studied in collaboration with Prof. Marc Van Hulle (Neural Computing) and Prof. Johan Wagemans (Experimental Psychology).
Thesis done by Bart Hamers
Speech recognition is probably one of the fields in AI that offers most possibilities to build industrial applications. Although there exists a wide variety of (professional) systems already, it still seems profitabel and certainly interesting to investigate new methods to tackle the problem of "finally making the computer understand what you're talking about". Self-organisation offers a lot of possibilities that are still waiting to be explored.
In previous studies, a "phoneme map" was developped that represented every phoneme by one position in the map. These positions in the map were first developped autonomously (Kohonen's self-organising map), after which they were labelled manually. One such cell would light up when its particular phoneme was detected. The trace that resulted from a spoken word, could be used to transcribe it to phonetic spelling. This system was called a "phonetic typewriter".
The objective of this thesis will be to investigate different self-organising principles to build a bank of filters to extract salient features from the training set (basically you uttering complete nonsense into a microphone) and to use this as a pre-processing step in a system that will either build a phoneme transcript or that will perform the complete task of speech recognition. The constituting filters will be developped such that they form a topographic map, where nearby positions in the input space are encoded by nearby positions in the map.
In the well-known nutshell that is normally used to inform the reader that the text is coming to an end, the thesis can be summarised in two steps. First, different self-organising principles should be examined (they are ready and waiting in the literature as we speek) and implemented. After this, a strategy has to be developped to actually USE these nice and cute algorithms. Various possibilities exist and even more have yet to be invented.
Thesis done by Johan Kaers
A general scheme for data mining based on neural networks is outlined. The requirements for this scheme were that it has to be applicable in real data environments, that it can run totally autonomously and that it is independent of every possible specific application. A scheme which satisfies these requirements can, when it is implemented, be plugged into a database management system (DBMS) as an independent, intelligent data mining module. The module is able to build autonomously a model of the data in the database, only using the intrinsic characteristics of the data and the information which is stored in the meta-database. When new records enter the database the data mining module keeps the data model up-to-date and consistent as such providing a basis for real applications such as classification, prediction, description, etc.
The data mining scheme consists of two parts, the first part performing principal component to do a dimensionality reduction in high dimensional and possibly sparse data spaces, the second part doing density-based clustering in order to build a density model and to partition the data into homogeneous groups.
The building blocks of the data mining scheme are neural networks. This choice is mainly motivated by their parallel and incremental nature, which makes them not only very suited for data mining tasks in huge databases, but also allows for updates of the data model without the need of rebuilding the whole model. However, a disadvantage of neural network learning is that the learning results are strongly dependent on the choice of initial parameter values and initial design decisions, which makes neural networks a difficult and even dangerous tool for an inexperienced user. Therefore, formulas and heuristics have to be provided in the data mining module such that it can search by itself for good initial values. In this thesis such formulas and heuristics are introduced for the neural network based PCA-stage.
A sequential mode of Sangers learning rule is proposed. The selection of Sangers learning rule is due to its robustness and due to the fact that it extracts the real principal components, which makes future updates easier when new dimensions have to be added or removed. A heuristic for determining the initial learning rates in the sequential mode together with an updating scheme have been developed. Test have been done to look for a minimal number of data points and epochs needed to build an acuurate data model. In a following section a method for intrinsic dimensionality determination, especially suited for incorporation in the sequential mode of Sangers learning rule, is introduced. This method is based on the measurement of the angle between the principal subspace extracted in former iteration steps and the last principal component. However, a significance test for this method still has to be developed. Finally, the issue of missing values is dealt with. We introduce an iterative search procedure for filling in missing values in a deficient pattern, based on the principal subspace. This method has the advantage (in contrast with mean substitution) that it makes use of the structure of the data to fill in the missing attributes. If the number of incomplete patterns is too large, the same procedure can also be used during principal component extraction, in order to make use of all available data.
Thesis done by Filip Deleus
For many purposes, not least for the exploration of data, and for its efficient storage and transmission, it is useful to reduce the dimensionality of the data. A classical technique in this regard is principal component analysis: the reduction of the data is achieved by projecting them onto a subspace whose basis vectors (principal components) are chosen in such a way that they maximize the variance of the projected data. The neural network field has contributed greatly by providing on-line (real-time) learning rules for obtaining these principal components. However, there are other ways by which an "interesting" set of basis vectors can be obtained. The general idea of projection pursuit is to define some other criterion by which an "interesting" set of basis vectors can be obtained, and to use a numerical technique to find the projection of most interest such as a neural network learning rule. Recently, a range of learning rules and corresponding neural networks were developed with which rather accurate models of the density distribution underlying the data can be built. Given these learning rules, and projection pursuit, a new approach to dimensionality reduction becomes feasible. The purpose of this thesis is to develop this combination of approaches and to apply it to e.g. (lossy) image compression.
Thesis done by Lukas
In topographic maps, nearby positions in the input space are encoded by nearby positions in the map. These maps, usually discrete networks of neurons, develop autonomously, i.e., following the principles of self-organization: local interactions between neurons of the map yield a globally, topographically ordered map. As a result of this technique, the input space is partitioned into non-overlapping partitionings, one for each neuron. Owing to the structured representation of the input data, this technique has enjoyed a wide range of applications such as clustering, pattern recognition, regression and dimensionality reduction, to name a few. Recently, a range of learning rules for self-organization were developed with which rather accurate models of the input density can be obtained. However, also in these learning rules, the input space is partitioned into uniform regions. In order to improve the performance of these rules, instead of using uniform regions, kernel-based neural activation regions should be used. The purpose of this thesis is to develop a new learning rule/self-organizationn principle for kernel-based topographic map formation, and to apply it to clustering analysis, and (unsupervised) classification or pattern recognition.
Thesis done by Pieter Op de Beeck
Neurons in the temporal cortex of monkeys are selective for complex images, i.e. they respond strongly to images of some but not all types of real objects. The purpose of this thesis is to determine how the activity of these neurons contribute to the categorization of the images into classes. More specifically, monkeys were trained to distinguish over 200 images of trees and of other real objects (non-trees). Furthermore, the activity of individual neurons was recorded during the tree/non-tree categorization. We would like to know if the stimulus-specific activity of the recorded neurons is sufficient enough to decide if a tree or a non-tree was shown. The analysis of the neural activity will be done with an artificial neural network (ANN) of which the input corresponds to the recorded neural activity and of which the output corresponds to the tree/non-tree decision made by the monkey. The ANN is supposed to learn the tree/non-tree distinction. After it has learned its task, the ANN is expected the deliver information about the necessary and sufficient conditions which the stimulus selectivities of the recorded neurons should possess in order to allow for the tree/non-tree categorization. The students will dispose of a data base of recorded cell activities as well as the necessary software for ANN learning.
Thesis done by Koen Moeyersons
Traditionally, topographic maps are considered as maps of the input space onto a discrete lattice. As a result of the topographic organization, nearby positions in the input space are encoded by nearby lattice positions. These maps develop autonomously, i.e. following the principles of self-organization, and yield an approximate, non-parametric model of the shape and extent of the input probability density. Recently, a new principle of self-organization (isostacy) has been discovered which relies on the use of fuzzy membership functions. The purpose of this thesis is to apply this technique to topographic map formation for the purpose of obtaining correct non-parametric models. Due to its adaptability, this learning principle is expected to provide new possibilities not only for probability density estimation in general, but also for clustering analysis (feature extraction) and discriminant analysis (classification) in particular.
Thesis done by Steven Raekelboom
In a natural environment, viewing a surface causes a distributed percept of local changes, which provides a description of physical surface properties such as roughness and shape. This percept is referred to as visual texture. The purpose of this thesis is to study and implement a neural network model that performs texture identification and segregation on real images automatically. In particular, the idea is to let the network develop all by itself the internal feature representations needed to perform this task. The network is able to adapt to changing environments by changing its connection weights. Based on the output of this network object recognition is performed.
Thesis done by Steven Sagaert
Thesis done by Temujin Gautama
The issues surrounding signal separation, or component discrimination, are founded on the temporal aspects of the input information. The introduction of time makes a static approach to the problem awkward and the consideration of dynamics necessary. Temporality remains an elusive property, and it is therefore, not surprising that a lot of engineering solutions exploit a transformation from the temporal to the spatial domain, as a form of preprocessing. Expermental evidence suggests that the brain also processes time by the same principle, by mapping time onto simultaneously available, spatially distributed properties of internal neural structures. The system proposed in this thesis, attempts to identify the minimal fucntional requirements, which are necessary to implement the low and high level processing capabilities of the brain. The hypothesis of the existence of a small set of basal functions and associated structures, which motivated this research, redefines the contribution of natural evolution along terms of function acquisition and structure compression.
Thesis done by Danny Martens
Traditionally, topographic maps are considered as maps of the input space onto a discrete lattice. As a result of the topographic organization, nearby positions in the input space are encoded by nearby lattice positions. These maps develop autonomously, i.e. following the principles of self-organization, and yield an approximate, non-parametric model of the shape and extent of the input probability density. Recently, a new principle of self-organization (isostacy) has been applied to topographic map formation and which yields correct non-parametric models. Due to its adaptability, this learning principle opens not only new possibilities for probability density estimation in general, but also for clustering analysis (feature extraction) and discriminant analysis (classification) in particular. Furthermore, since simulations have shown that this new learning principle is similar to the one used in Hidden Markov Models (HMM), it is evident that it could be used as a pre-processor for HMMs. The latter are commonly used in speech recognition. The aim of this thesis is to study the application of the aforementioned principle to clustering- and discriminant analysis and its use as a pre-processor module ("probability machine") in hybrid systems.
Thesis done by Bart Buelens
Back to the working-group home page