学术报告

题      目：Galerkin Transformer

报  告  人：Shuhao Cao 博士 讲师    (邀请人：钟柳强 )

                                 Washington University in St. Louis

时      间：2022-06-07  10：00-11：00

腾讯会议：727-510-781  密码：220607

摘      要：

       Transformer in "Attention Is All You Need" is now the ubiquitous architecture in every state-of-the-art model in Natural Language Processing (NLP), Computer Vision (CV), and scientific breakthroughs such as AlphaFold 2 by Deepmind. At its heart and soul is the "attention mechanism", we shall learn the mathematical structure of the attention for the first time via the lens of the operator approximation theory in Hilbert spaces. Inspired by finite element methods, the attention mechanism can be interpreted as a learnable Petrov-Galerkin projection. After being further modified and combined with the best operator, Galerkin Transformer achieves a generational leap of its performance in various PDE-related operator learning tasks.