Introduction to Statistical Machine Learning 2009 (ANU COMP6467/4670)
Course Coordinators
Presenter
Tutor
Time and Place
- Lectures
- Tutorials
- Thursday 1pm - 3pm, Building 108, CSIT, N113
- For more detailed information, please refer to the schedule at the end of this page.
News
- 31-05-2009: Added all slides in one file.
- 08-05-2009: Oral examination date: 16 June 2009
- 31-05-2009: Code for Viterbi algorithm (tutotrial 7) is posted.
- 05-05-2009: Assignment 2 is posted.
- 12-04-2009: Sample code for neural network is posted.
- 23-03-2009: Assignment 1 is posted.
- 1-03-2009: Changed Wednesday lecture room to R221 (same as Tuesday).
- 1-03-2009: Lecture slides are online.
- 22-02-2009: Changed written to oral exam.
- 19-02-2009: Changed Wednesday lecture room to N329, Aquarium.
- 29-01-2009: Added movie to show Elefant in action.
- 28-01-2009: Released a new version of Elefant (v 0.3). We will use this version in the course.
- 12-01-2009: Setting up the web site
Enrollment
Prerequisites: some background in elementary statistics and probabilities, numerical algorithms, and programming experience.
If you fulfil official requirements, please send an email to Christfried Webers (firstname.lastname@nicta.com.au) that he can support your enrolment.
Assessment
- 2 written assignments (20% each)
- oral examination (60%)
Assessment Late Policy
- For every beginning day after the deadline of an assignment, the mark will be reduced by 20%.
Important dates and special announcements
- Assignment 1 using Fisher Iris Data , due: April 27, 23:59 (released: March 23)
- Assignment 2 , due: May 24, 23:59 (released: May 5)
- Final examination: 16 June 2009
Contact hours for students
After the lecture and per email
Textbook
Required:
We also recommend (ordered by priority):
- Hastie, Tibshirani, and Friedman: The Elements of Statistical Learning, Springer
- MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press
- Hutter, Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability, Springer
Overview
Statistical Machine Learning plays a key role in science and technology. Some examples of applications using Statistical Machine Learning techniques are e.g.
- e-mail spam filtering,
- web page ranking,
- handwritten ZIP code recognition,
- identification of risk factors for cancer,
- object recognition in computer vision, and,
- autonomous robot navigation.
Some of the basic questions raised are
- What is a good model for the available data?
- How computationally effective can the parameters of the model be fitted to the available data?
- How does a model perform on future data?
This course provides a broad but thorough introduction to the methods and practice of statistical machine learning. Topics covered will include Bayesian inference and maximum likelihood modeling; regression, classification, density estimation, clustering, principal and independent component analysis; parametric, semi-parametric, and non-parametric models; basis functions, neural networks, kernel methods, and graphical models; deterministic and stochastic optimisation; overfitting, regularisation, and validation.
The course employs the Elefant - Machine Learning Toolbox developed at NICTA and written in the programming language Python. The graphical user interface of Elefant and a number of predefined building blocks (e.g. data readers and writers, algorithms) allow to easily setup and store a Machine Learning experiment. Elefant is Open Source and available for Linux, Mac OS X, and Windows. It will be provided on all computers used in the tutorials and labs. (Please watch a movie demonstrating Elefant in action.)
Schedule
(to be adapted and refined throughout the course)| week | Lecture | Lecture | Tutorial/Lab | Tue 4 - 5.30 pm | Wed 4 - 5.30 pm | Thu 1 - 3 pm | Ian Ross, R221 Graduate Teaching Room | Ian Ross, R221 Graduate Teaching Room | CSIT, N113 | 23/2 - 27/2 | Overview (Slides) | Introduction (Slides) | none | 2/3 - 6/3 | Probability (Slides) | Linear Algebra (Slides) | Optimisation (see also "From Matlab to Python/Numpy") | 9/3 - 13/3 | Linear Regression (Slides) | Linear Regression (Slides) | Sample code for Optimisation | 16/3 - 20/3 | Linear Classification (Slides) | Linear Classification (Slides) | Regression ( Sample code) | 23/3 | Assignment 1 using Fisher Iris Data | 23/3 - 27/3 | Neural Networks (Slides) | Neural Networks (Slides) | Classification using Fisher Iris Data | 30/3 - 3/4 | Kernel Methods (Slides) | Sparse Kernel Machines (Slides) | Neural Network | 6/4 - 10/4 | Graphical Models (Slides) | Graphical Models (Slides) | Sample Code for Neural Network | 13/4 - 17/4 | none | none | none | 20/4 - 24/4 | none | none | none | 27/4 - 1/5 | Graphical Models (Slides) | Mixture Models and EM (Slides) | Kernels and Elefant (manual) | 5/5 | Assignment 2 using Fisher Iris Data | 4/5 - 8/5 | Mixture Models and EM (Slides) | Approximate Inference (Slides) | 11/5 - 15/5 | Sampling (Slides) | Principal Component Analysis (Slides) | Return Assignment 1; Oral Exam Preparation | 18/5 - 22/5 | Sequential Data (Slides) | Sequential Data (Slides) | Eigenfaces | 25/5 - 29/5 | Combining Models (Slides) | Selected Topics (Slides) | Hidden Markov Model ( Sample code ) | 1/6 - 5/6 | Discussion/Summary (Slides) | Exam preparations | none | 16/6 | Oral Examination (All Slides) |
|---|
For excellent supplemental slides, see Andrew Moore's Tutorials.
Last modified 2009-06-08 14:13