Let $A$ be a square matrix. The resolvent, $(A-z I)^{-1}, z \in \mathbb{C}$, plays an important role in many applications; for example, in studying functions of $A$, one often uses the Cauchy integral formula,$$f(A)=-\frac{1}{2 \pi i} \int_{\Gamma}(A-z I)^{-1} f(z) d z$$where $\Gamma$ is the boundary of a region $\Omega$ that contains the spectrum of $A$ and on which $f$ is analytic. If $z$ is very close to a simple eigenvalue $\lambda$ of $A$ - much closer to $\lambda$ than to any other eigenvalue of $A$ - then $(A-z I)^{-1} \approx \frac{1}{\lambda-z} x y^*$, where $x$ and $y$ are right and left normalized eigenvectors of $A$ corresponding to eigenvalue $\lambda$. It is sometimes observed, however, that $(A-z I)^{-1}$ is close to a rank one matrix even when $z$ is not very close to an eigenvalue of $A$. In this case, one can write $(A-z I)^{-1} \approx \sigma_1(z) u_1(z) v_1(z)^*$, where $\sigma_1(z)$ is the largest singular value of $(A-z I)^{-1}$ and $u_1(z)$ and $v_1(z)$ are the corresponding left and right singular vectors. We use singular value/vector perturbation theory to describe conditions under which $(A-$ $z I)^{-1}$ can be well-approximated by rank one matrices for a wide range of $z$ values. If $\lambda$ is a simple ill-conditioned eigenvalue of $A$, if the smallest nonzero singular value of $A-\lambda I$ is well-separated from 0 , and if a certain other condition involving the singular vectors of $A-\lambda I$ is satisfied, then it is shown that $(A-z I)^{-1}$ is close to a rank one matrix for a wide range of $z$ values. An application of this result in comparing bounds on $\|f(A)\|$ is described [1] for example, in studying functions of $A$, one often uses the Cauchy integral formula,$$f(A)=-\frac{1}{2 \pi i} \int_{\Gamma}(A-z I)^{-1} f(z) d z$$where $\Gamma$ is the boundary of a region $\Omega$ that contains the spectrum of $A$ and on which $f$ is analytic. If $z$ is very close to a simple eigenvalue $\lambda$ of $A$ - much closer to $\lambda$ than to any other eigenvalue of $A$ - then $(A-z I)^{-1} \approx \frac{1}{\lambda-z} x y^*$, where $x$ and $y$ are right and left normalized eigenvectors of $A$ corresponding to eigenvalue $\lambda$. It is sometimes observed, however, that $(A-z I)^{-1}$ is close to a rank one matrix even when $z$ is not very close to an eigenvalue of $A$. In this case, one can write $(A-z I)^{-1} \approx \sigma_1(z) u_1(z) v_1(z)^*$, where $\sigma_1(z)$ is the largest singular value of $(A-z I)^{-1}$ and $u_1(z)$ and $v_1(z)$ are the corresponding left and right singular vectors.We use singular value/vector perturbation theory to describe conditions under which $(A-$ $z I)^{-1}$ can be well-approximated by rank one matrices for a wide range of $z$ values. If $\lambda$ is a simple ill-conditioned eigenvalue of $A$, if the smallest nonzero singular value of $A-\lambda I$ is well-separated from 0 , and if a certain other condition involving the singular vectors of $A-\lambda I$ is satisfied, then it is shown that $(A-z I)^{-1}$ is close to a rank one matrix for a wide range of $z$ values. An application of this result in comparing bounds on $\|f(A)\|$ is described [1].[-]

In an era where Artificial Intelligence (AI) is permeating virtuallly every single field of science and engineering, it is becoming critical to members of the numerical linear algebra community to understand and embrace AI , and to contribute to its advancement, and more broadly to the advancement of machine learning. What is fascinating and rather encouraging is that Numerical Linear Algebra (NLA) is at the core of machine learning and AI. In this talk we will give an overview of Deep Learning with an emphasis on Large Language Models (LLMs) and Transformers [3, 4]. The very first step of LLMs is to convert the problem into one that can he exploited by numerical methods, or to be more accurate, by optimization techniques. All AI methods rely almost entirely on essentially 4 ingredients: data, optimization methods, statistical intuition, and linear algebra. Thus, the first task is to map words or sentences into tokens which are then imbedded into Euclidean spaces. From there on, the models refer to vectors and matrices. We will show a few examples of important developments in ML, that were heavily based on linear algebra ideas. Among these, we will briefly discuss LoRa [1] a technique in which low-rank approximation was used to reduce computational cost in some models, leading to gains of a few orders of magnitude. Another contribution that used purely algebraic arguments and that had a major impact on LLMs is the article [2]. Here the main discovery is that the nonlinear ""self-attention"" in LLMs can be approximated linearly, resulting in huge savings in computations, as the computational complexity was decreased from $O\left(n^2\right)$ to $O(n)$.The talk will be mostly a survey of known recent methods in AI with the primary goal of unraveling the mathematics of Transformers. A secondary goal is to initiate a discussion on the issue of how NLA specialitst can participate in AI research.[-]