勷勤数学•专家报告

题      目：qNBO: quasi-Newton Meets Bilevel Optimization

报  告  人： 刘勇进 教授  (邀请人：陈艳男)

                                              福州大学

时      间： 9月20日  13：30-14:30

地     点：数科院东楼二楼会议室

报告人简介:

          刘勇进，福州大学嘉锡学者特聘教授、博士生导师，福建省闽江教育领军人才闽江特聘教授，担任福建省应用数学中心（福州大学）主任。研究兴趣主要包括：最优化理论、方法与应用，大规模数值计算，统计优化等，研究成果在包括Mathematical Programming (Series A)、SIAM Journal on Optimization、SIAM Journal on Scientific Computing等优化与计算领域国际顶级学术期刊上发表。主持国家重点研发计划项目课题1项，主持国家自然科学基金5项（面上项目4项、青年基金1项），主持教育部、省重点项目等部省级纵向科研项目7项。现任中国数学会理事、中国运筹学会理事、中国运筹学会数学规划分会常务理事、中国运筹学会算法软件与应用分会常务理事、中国统计学会理事、福建省运筹学会会长、福建省数学学会副会长。

摘      要：

           Bilevel optimization, addressing challenges in hierarchical learning tasks, has gained significant interest in machine learning. The practical implementation of the gradient descent method to bilevel optimization encounters computational hurdles, notably the computation of the exact lower-level solution and the inverse Hessian of the lower-level objective. Although these two aspects are inherently connected, existing methods typically handle them separately by solving the lower-level problem and a linear system for the inverse Hessian-vector product. In this talk, we introduce a general framework to address these computational challenges in a coordinated manner. Specifically, we leverage quasi-Newton algorithms to accelerate the resolution of the lower-level problem while efficiently approximating the inverse Hessian-vector product. Furthermore, by exploiting the superlinear convergence properties of BFGS, we establish the non-asymptotic convergence analysis of the BFGS adaptation within our framework. Numerical experiments demonstrate the comparable or superior performance of the proposed algorithms in real-world learning tasks, including hyperparameter optimization, data hyper-cleaning, and few-shot meta-learning.

          欢迎老师、同学们参加、交流！