Optimization in Machine Learning Ph.D. Course by İlker Birbil

Optimization in Machine Learning

Ph.D. Course by İlker Birbil

22-23 May 2023

This course consists of five lectures about the important role of optimization in machine learning. Each lecture starts with a brief introduction to one of the supervised learning methods. The latter part of the lecture is devoted to optimization algorithms that are used in model training. Preferring an optimization algorithm over others is not an easy task. Thus, the selection is done by a simple principle: The selected optimization algorithm should either be implemented in a well-known software package, or it should be something that brings in a new perspective to the subject.

Here are the five lectures:

• Linear Models and Regularization: linear, ridge, lasso, elastic net, logistic regression
• Support Vector Machines: primal and dual models, kernel trick
• Neural Networks: backpropagation, stochastic gradient descent, and its variants
• Boosting: margin maximization, gradient boosting, relation to duality
• Trees and Forests: optimal classification trees, subset selection, rule generation

Prerequisites:

• Proficiency in linear algebra, probability, and multivariable calculus
• Working knowledge of linear, nonlinear, and integer optimization

Teaching material:

• Lecture notes
• Research papers

Registration: Please fill in this form.

Address: Altunizade Digital Campus, Sabancı University