勷勤数学•专家报告

题      目：Nonconvex Nonsmooth Multicomposite Optimization and Its Applications to Recurrent Neural Networks

报  告  人：金玲子 博士  (邀请人：王晓宙)

                                              香港理工大学

时      间： 10月16日  11：00-12:00

地     点：数科院东楼401

报告人简介:

          博士毕业于香港理工大学，导师为陈小君教授，副导师为王晓教授。现于香港理工大学从事博士后研究工作。主要研究最优化理论和其在机器学习中的应用，包括机器学习中的非凸非光滑优化算法、随机优化算法、大规模非凸优化问题、约束优化问题。已在 Math. Comp. 及 SIAM J. Optim. 等顶级学术期刊上以第一作者发表论文。

摘      要：

         We consider a class of nonconvex nonsmooth multicomposite optimization problems where the objective function consists of a Tikhonov regularizer and a composition of multiple nonconvex nonsmooth component functions. Such optimization problems arise from tangible applications in machine learning and beyond. To define and compute its first-order and second-order d(irectional)-stationary points effectively, we establish its equivalence with the corresponding constrained and $\ell_{1}$-penalty reformulations in terms of global optimality and d-stationarity. Furthermore, we establish easily verifiable conditions under which an approximate first-order d-stationary point of the problem is guaranteed to be an approximate second-order d-stationary point. Subsequently, we propose a sequential regularized piecewise affine algorithm, SRPA, for the $\ell_{1}$-penalty reformulation. We establish the global convergence of SRPA to second-order d-stationary points and the complexity bound of $\mathcal{O}(\epsilon^{-2})$ for obtaining an $\epsilon$-approximate second-order d-stationary point. Finally, numerical experiments for training RNNs (Recurrent Neural Networks) on synthetic and real-world datasets demonstrate the promising performance of SRPA compared with state-of-the-art algorithms.

          欢迎老师、同学们参加、交流！