Course Notes of Machine Learning

From Dahuawiki

This page organizes notes that I take for the MIT Machine Learning course. After each lecture, I will update the notes here. This way on one hand facilitates my review of the course materials, on the other hand contributes to the community by the sharing what I learned from MIT, the most prestigious institute in the world.

Introduction

Machine Learning is a subject to study how to predict based on past experience. There are typically two stages in a machine learning problem. It learns a model or prediction rule based on known samples, which is then applied to unseen samples to give a prediction.

Different types of learning

Based on different types of information available in the training stage, learning methods can be generally categorized into following classes:

Supervised learning

pairs of corresponding input and output are given to learn the prediction

Unsupervised learning

only input samples are given to learn the model of the samples

Semi-supervised learning

the learning process makes use of both labeled samples as well as unlabeled samples

Reinforcement learning

learn the rule to guide future actions based on the feedback from practice

Generalization and model selection

The task of learning is to learn a prediction rule that works well on the samples unseen in the training stage. Formally, we are to find the prediction rules that generalize well.

In order to find predictors that generalize well, the key is to constrain the domain from which we derive the predictor, or in other words, the class of prediction functions. The class should not be too large such that we tend to find the functions that work perfect on training set but generalize poorly. It should not be too small either, otherwise, we may be in the risk of failing to find any function that work well even on the training samples. Finding a proper class of functions is a key problem in machine learning, which is known as model selection.

Topic List

The list is given based on the course syllabus, which is subject to changes.

1. Linear classification and Perceptron
2. Maximum margin classifier and Support vector machine
3. Logistic regression
4. Linear regression and Ridge regression
5. Kernels and Kernel regression
6. Kernel SVM
7. Model selection
8. Description length and feature selection
9. Combining classifiers
10. Boosting
11. Margin and Generalization
12. Mixtures and the EM algorithm
13. Regularization
14. Clustering
15. Spectral clusering
16. Markov models
17. Hidden Markov Models
18. Bayesian networks
19. Probabilistic inference
20. Current problems in machine learning

For mid-term

Mid-term-exam-points

Chunk-based Faculty Biography Parsing and Automatic Resume Generation