Computing invariant subspaces is at the core of many applications, from machine learning to signal processing, and control theory, to name just a few examples. Often one wishes to com- pute the subspace associated with eigenvalues located at one end of the spectrum, i.e., either the largest or the smallest eigenvalues. In addition, it is quite common that the data at hand undergoes frequent changes and one is required to keep updating or tracking the target invariant subspace. The talk will present standard tools for computing invariant subspaces, with a focus on methods that do not require solving linear systems. One of the best known techniques for computing invariant subspaces is the subspace iteration algorithm [2]. While this algorithm tends to be slower than a Krylov subspace approach such as the Lanczos algorithm, it has many attributes that make it the method of choice in many applications. One of these attributes is its tolerance of changes in the matrix. An alternative framework that will be emphasized is that of Grassmann manifolds [1]. We will derive gradient-type methods and show the many connections that exist between different viewpoints adopted by practitioners, e.g., the TraceMin algorithm [3]. The talk will end with a few illustrative examples.[-]

In an era where Artificial Intelligence (AI) is permeating virtuallly every single field of science and engineering, it is becoming critical to members of the numerical linear algebra community to understand and embrace AI , and to contribute to its advancement, and more broadly to the advancement of machine learning. What is fascinating and rather encouraging is that Numerical Linear Algebra (NLA) is at the core of machine learning and AI. In this talk we will give an overview of Deep Learning with an emphasis on Large Language Models (LLMs) and Transformers [3, 4]. The very first step of LLMs is to convert the problem into one that can he exploited by numerical methods, or to be more accurate, by optimization techniques. All AI methods rely almost entirely on essentially 4 ingredients: data, optimization methods, statistical intuition, and linear algebra. Thus, the first task is to map words or sentences into tokens which are then imbedded into Euclidean spaces. From there on, the models refer to vectors and matrices. We will show a few examples of important developments in ML, that were heavily based on linear algebra ideas. Among these, we will briefly discuss LoRa [1] a technique in which low-rank approximation was used to reduce computational cost in some models, leading to gains of a few orders of magnitude. Another contribution that used purely algebraic arguments and that had a major impact on LLMs is the article [2]. Here the main discovery is that the nonlinear ""self-attention"" in LLMs can be approximated linearly, resulting in huge savings in computations, as the computational complexity was decreased from $O\left(n^2\right)$ to $O(n)$.The talk will be mostly a survey of known recent methods in AI with the primary goal of unraveling the mathematics of Transformers. A secondary goal is to initiate a discussion on the issue of how NLA specialitst can participate in AI research.[-]