IE-OPIM Joint Graduate Seminar: Ş. İlker Birbil (Sabancı University)
Joint Work with: U. Şimşekli, H. Koptagel, F. Öztoprak, A. T. Cemgil
“Distributed Incremental Optimization Algorithm Using Quadratic Approximations for Partially Separable Problems”
Date: Wednesday, September 16, 2015
Time: 13:40 – 14:30
Location: FENS G032
Abstract: We propose HAMSI, a provably convergent incremental algorithm for solving large-scale partially separable optimization problems that frequently emerge in machine learning and inferential statistics. The algorithm is based on a local quadratic approximation and hence allows incorporating a second order curvature information to speed-up the convergence. Furthermore, HAMSI needs almost no tuning, and it is scalable as well as easily parallelizable. In large-scale simulation studies with the MovieLens datasets, we illustrate that the method is superior to a state-of-the-art distributed stochastic gradient descent method in terms of convergence behavior. This performance gain comes at the expense of using memory that scales only linearly with the total size of the optimization variables. We conclude that HAMSI may be considered as a viable alternative in many scenarios, where first order methods based on variants of stochastic gradient descent are applicable.
Bio: Ş. İlker Birbil received his PhD degree from North Carolina State University, Raleigh, USA. He then worked for two years as a postdoctoral research fellow in the Netherlands. His research interests include large-scale local and global nonlinear optimization with particular emphasis on algorithm development. He is also an associate editor of Journal of Industrial and Management Optimization (JIMO). Currently, he is working as a faculty member in Sabancı University, Istanbul, Turkey, where he teaches various courses on operations research.