Many problems in real-world applications of machine learning can be formalized as classical statistical problems, e.g., pattern recognition, regression or dimension reduction, with the caveat that the data are often not vectors of numbers. For example, protein sequences and structures in computational biology, text and XML documents in web mining, segmented pictures in image processing, or time series in speech recognition and finance, have particular structures which contain relevant information for the statistical problem but can hardly be encoded into finite-dimensional vector representations.
Kernel methods are a class of algorithms well suited for such problems. Indeed they extend the applicability of many statistical methods initially designed for vectors to virtually any type of data, without the need for explicit vectorization of the data. The price to pay for this extension to non-vectors is the need to define a so-called positive definite kernel function between the objects, formally equivalent to an implicit vectorization of the data. The "art" of kernel design for various objects have witnessed important advances in recent years, resulting in many state-of-the-art algorithms and successful applications in many domains.
The goal of this course is to present the mathematical foundations of kernel methods, as well as the main approaches that have emerged so far in kernel design. We will start with a presentation of the theory of positive definite kernels and reproducing kernel Hilbert spaces, which will allow us to introduce several kernel methods including kernel principal component analysis and support vector machines. Then we will come back to the problem of defining the kernel. We will present the main results about Mercer kernels and semigroup kernels, as well as a few examples of kernel for strings and graphs, taken from applications in computational biology, text processing and image analysis. Finally we will touch upon topics of active research, such as large-scale kernel methods and deep kernel machines.
Lecture (in english) usually take place at Campus Jourdan 48 bd Jourdan Paris 14, in amphi Jourdan, 1:30-4pm.
The first class on January 16th will take place at ENS Cachan, amphi Curie
The exam will take place at MINES ParisTech, 60 boulevard Saint-Michel, 75006 Paris (RER Luxembourg). Check your room here.
| Date | Place | Lecturer | Topic | Slides | More material |
| Jan 16 | ENS Cachan, amphi Curie | JM | Positive definite kernel, RKHS, Aronszajn's theorem | 1-46 | Uniqueness of the RKHS Aronszajn's theorem |
| Jan 23 | ENS Campus Jourdan, amphi Jourdan | JM | Kernel trick, Representer theorem, kernel ridge regression | 47-103 | |
| Jan 30 | ENS Campus Jourdan, amphi Jourdan | JM | Supervised classification, Kernel logistic regression, large margin classifiers, SVM | 103-164 | Bartlett et al., 2006 |
| Feb 6 | ENS Campus Jourdan, amphi Jourdan | JM | Unsupervised learning, kernel PCA, kernel-Kmeans, spectral clustering, kernel CCA | 165-201 | |
| Feb 13 | ENS Campus Jourdan, amphi Jourdan | JPV | Green, Mercer, Herglotz, Bochner kernels | 202-277 | |
| Feb 20 | ENS Campus Jourdan, amphi Jourdan | JPV | Kernels from generative models, kernels for sequences | 298-367 | |
| Feb 27 | Break | ||||
| Mar 6 | ENS Campus Jourdan, amphi Jourdan | Arthur Gretton (UCL) | Hypothesis testing and MMD | Part 1, Part 2s | |
| Mar 13 | ENS Campus Jourdan, amphi Jourdan | JPV | Graph kernels | 403-503 | |
| Mar 20 | ENS Campus Jourdan, amphi Jourdan | JPV | MKL, large-scale kernel methods | 517-588 | |
| Mar 27 | MINES ParisTech, room L108-L106-L213 | 2pm-4pm: Final exam |