Machine Learning

From Dahuawiki

What is Machine Learning

Machine Learning is an important field in artificial intelligence, which mainly concerns how a computer can learn from observations and practice and thereby make predictions for new inputs.

Machine Learning is based on theoretical computation theory and statistics. The principles, models, and algorithms are used by a wide range of artificial intelligence fields like computer vision, speech recognition, pattern classification, natural language processing, data mining, and bioinformatics.

Machine Learning comprises a number of areas. Here is a partial list of the active topics in the learning literatures

1. Classification and Regression
2. Clustering
3. Decision Trees
4. Kernel Density Estimation
5. Support Vector Machines
6. Neural Networks
7. Bayesian Networks
8. Markove Random Field and Conditional Random Field
9. Gaussian Process
10. Belief Propagation
11. Monte Carlo and Sampling
12. Graphical Models
13. Dimension Reduction
14. Kernel-based Learning
15. Manifold-based Learning
16. Graph-based Inference
17. Semi-supervised Learning and Transduction
18. Model Fusion
19. Model Selection
20. Information Theory based Learning
21. Multi-Instance Learning

Different Types of Learning

Depending on the forms of input and output, the learning process can be roughly categorized into the following types

• Supervised Learning is to learn the map from input to output. Pairs of input and output are given in the learning stage.
• Unsupervised Learning aims at modeling the input data, such as their clustering structure and sample distribution.
• Semi-supervised Learning takes advantages of both labeled and unlabeled data.
• Transductive Learning makes prediction on new samples based on the labeled samples.
• Active Learning tries to gain information by asking "supervisor" selected questions.
• Reinforcement Learning learns the strategy of action from the feedback of environment.

Recommended Books

The following books are recommended to the researchers in machine learning.

• J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
• M. I. Jordan. Learning in Graphical Models. MIT Press, 1998.
• Vladimir N. Vapnik. Statistical Learning Theory. Wiley-Interscience, 1998.
• Richard O. Duda, Peter E. Hart, and David G. Stork. Pattern Classification (2nd ed.). Wiley-Interscience, 2000.
• Bernhard Schlkopf and Alexander J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, 2001.
• Andrew R. Webb. Statistical Pattern Recognition (2nd ed.). John Wiley and Sons Ltd. 2002.
• Olivier Chapelle, Bernhard Schölkopf, and Alexander Zien. Semi-Supervised Learning. MIT Press, 2006.

Course

The course Machine Learning (6.867) at MIT OpenCourseWare (OCW) taught by Tommi Jaakkola is a nice introduction to the field of machine learning.

I am now taking this course at MIT. The notes that I took are shared in this Wiki in Course Notes of Machine Learning.

Conferences

• Annual Conference on Neural Information Processing Systems (NIPS)
• International Conference on Machine Learning (ICML)
• National Conference on Artificial Intelligence (AAAI)
• International Joint Conferences on Artificial Intelligence (IJCAI)
• Uncertainty in Artificial Intelligence Conference (UAI)