DD3364: Elements of Statistical Learning (9 points)
This PhD level course will be given in English in the Spring of 2012.
Description of the course
This graduate course is based on the book Elements of Statistical Learning (second edition) by Trevor Hastie, Robert Tibshirani and Jerome Friedman, 2009. The authors are respected professors of statistics at Stanford University whose work stands at the crosswords of machine learning, data-mining and statistics. Their book presents a statistical interpretation and basis for many of the most popular and successful machine learning algorithms used today. It has a nice balance between readability, detail and insight for the mathematically and statistically proficient - people with degrees based in engineering, physics, computer science etc. - and is very accessible with a firm eye kept on practical concerns. Therefore it is an ideal textbook for doctoral students who already use machine learning algorithms within their own research, but would like to solidify their understanding of these techniques. The course will focus on the book's material covering supervised learning. Ideally, after the course students will be acquainted with the basics of modern machine learning and will have deepened and clarified their understanding of the area.
The workload for the student will involve mandatory assigned readings, and attendance of lectures and optional pen & paper and programming exercises. The credits will be awarded based on completion of the assigned readings and attendance of the lectures (3) and completion of the pen & paper exercises (3), programming exercises (3).
Course DetailsNote the course will be broken up into two parts with Part I covering the first half of the book and Part II the second half. See the schedule for a more detailed description of the content of each part.
ScheduleThe course schedule is available here: Schedule.
How you'll learnThe aim of this course is to facilitate you reading and understanding the content of the course textbook. As such my vision of how this process will happen is that you obtain
AssessmentThis is a PhD level course and there will be no final exam. To pass the course and acquire 3 credits you must complete the assigned readings and attend the lectures. While for the other credits you'll have to hand in pen & paper exercises (3 credits) and complete programming exercises and experiments (3 credits) (ideally you will tailor these to data-sets relevant to your research). More details of the latter will appear as we get nearer the start of the course. It is also my intention to run, with the help of the students, (optional) exercises session in between lectures to help with problem solving and programming projects/assignments.
Mailing list and RegisteringIf you are interested receiving e-mails about the course please mail me. If you want to register for the course please fill in the form web-form. You should fill it out with your details once I give you the password at the second lecture.
The course will use the book Elements of Statistical Learning (second edition) by Trevor Hastie, Robert Tibshirani and Jerome Friedman, 2009. This is available for download on-line, but it is perhaps recommended that students buy it.
Students may also find the book Modern Multivariate Statistical Techniques Regression, Classification, and Manifold Learning by Alan Julian Izenman an insightful companion to the main course book for some of the topics covered. It goes into greater depth on some of the issues.
I also recommend the The Matrix Cookbook by Kaare Brandt Petersen and Michael Syskind Pedersen. It contains a large list of the matrix identities and and how to differentiate with respect to vectors, matrices etc. Basically, it is invaluable when you trying to work out calculations.
Useful software packages
PrerequisitesStudents taking the course should be relatively proficient with the topics of linear algebra, statistics. It would also greatly help if the students have previously taken basic courses in machine learning or are at least aware of some of the issues.