Description:
This course aims at providing a foundation on machine learning, from the perspective of training a regression model to perform data analysis. Students completing the course will be able to mathematically formulate an engineering problem as a learning problem, by setting the appropriate loss function, training data, testing data, and perform simple training schemes. Topics covered in this course will serve as the foundation for further subjects including classification and learning theory.
Topics Covered:
- Review of linear algebra: To review basic concepts in linear algebra including norm, inner product, linear combination, basis functions, matrix-vector multiplication, eigen-decomposition
- Principles of regression: To understand the basic principles of regression, including loss function, linear models, parameter update.
- Solving a regression problem: To derive the linear least squares solutions and understand the properties under-determined and over-determined linear systems.
- Regularization techniques: To apply ridge regression techniques and LASSO regression techniques.
- Optimization: To review constrained and unconstrained minimization, Lagrange multiplier, convexity, gradient descent, and stochastic gradient descent