Research Statement

Summary

My original mathematical interests were formed by my advisors belonging to the Soviet mathematical elite: Dyakonov and Lebedev, both students of Sobolev at the Moscow State University. Renowned mathematicians Kublanovskaya and academician Godunov were my Ph.D. committee members in 1985. Until 1992, I was a researcher at Marchuk Institute of Numerical Mathematics of the Soviet (now Russian) Academy of Sciences, and taught mathematics part-time in top Soviet universities. Jointly with academician Bakhvalov, I solved the problem of slow convergence of iterative methods for PDEs and linear elasticity with highly contrast materials, typical in homogenization.??Jointly with Widlund of the NYU Courant Institute, I applied domain decomposition techniques to prove regularity and derive pioneering FEM accuracy estimates in standard Sobolev norms for?elliptic?partial differential equations, both results uniform in jumps in PDE coefficients. My other influential collaboration, with Osborn of U. Maryland, resulted in sharp explicit error bounds for Ritz FEM method in eigenvalue problems.

Supported by National Science Foundation and?Department of Energy?grants for research in numerical PDEs and matrix analysis, I was a Professor of?Mathematics?at the?University of Colorado Denver in 1994-2014, awarded for excellence in research and teaching. I published over hundred papers and exposed mathematics to thousands of undergraduate and hundreds of graduate students over three decades in academia. I co-organized three Oberwolfach meetings.

Jointly with my six PhD CU-Denver students, I advanced majorization to the Rayleigh–Ritz method and angles between flats, e.g., deriving first sharp bounds for changes in Ritz values. My papers on angles between subspaces became standard references in applications, e.g., signal processing, communications, computer vision, and machine learning.?My most famous and influential work was creation of the Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) matrix-free iterative eigenvalue solver in 2001, and exposition of preconditioning for eigenvalue problems to matrix analysts, parallel software developers, and engineers. My collaboration with physicists resulted in wide acceptance of LOBPCG in public software for electromagnetics and material sciences, where in many cases it outperforms classical Lanczos methods. I pioneered preconditioning for spectral clustering, and applied LOBPCG for image segmentation and denoising.

My industrial mathematician career started with Mitsubishi Electric Research Laboratories (MERL) in 2012-2018. At MERL, I invented algorithms for control, data mining, computer vision, coding, communications, and signal processing, having fourteen US patent applications filed (ten currently issued) and over twenty papers, contributing to state-of-the-art mathematical modeling, e.g.,:

• ab initio DFT simulations on AWS and other clouds using ABINIT and Siesta to accurately calculate band gaps and offsets of semiconducting materials used in optical devices,
• Big Data mining of time series for anomaly detection assisted by spectral clustering,
• ?invented algorithms for model predictive control (MPC) with automotive and transportation applications: preconditioned algorithms for nonlinear Newton-Krylov-based MPC, multiple-precision systems and preconditioning in active set and interior point techniques for?MPC,
• ?developed Krylov methods for graph-based signal denoising and up-sampling with applications to multimode image/video fusion and super-resolution,
• invented systems for multi-exponent data from measurements to analyze traps in semiconductor devices.

I analyzed?stock portfolio optimization and control techniques for option pricing, consulting for Capital Fund Management (CFM) in 2018.

I invented algorithms for quantum computing and co-authored two patent applications, currently published, consulting for Zapata Computing in 2018-2019.

I developed algorithms for embedded and backend applications of real-time big data time series analysis with emphasis on anomaly detection assisted by clustering with applications in automotive and transportation industries at Cognomotiv in 2019-2020.

My present research interests concern algorithms and?architecture design of Silicon Photonics chips for high-performance computing, ML, and AI with applications in data analytics, physics, and computational finance.

I contribute to public software at GitHub, e.g., for partial SVD and PCA in scikit-learn, and lobpcg in SciPy.

Review of Selected Results and Inventions

Iterative algorithms for linear systems, eigen- and singular value problems, and constrained optimization is my core area of expertise. Preconditioning is the key for efficiency to accelerate convergence of iterations. My main research contributions included:

• inventing the Locally Optimal Block Preconditioned Conjugate Gradient?(LOBPCG) that is a?matrix-free method?for finding the largest (or smallest)?eigenvalues?and the corresponding?eigenvectors?of a symmetric positive definite?generalized eigenvalue problem. LOBPCG can be trivially adopted for computing several largest?singular values?and the corresponding singular vectors (partial SVD), e.g., for iterative computation of PCA, without explicitly computing the?covariance?matrix.?LOBPCG has become one of de facto standard algorithms, competing with and having several advantages, e.g., direct preconditioning and blocking, over the classical Lanczos method.
• implementing LOBPCG in many public software packages:?Octave,?MATLAB, SciPy, scikit-learn, ABINIT, Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX)?with interfaces to?PETSc/SLEPc,?hypre, and PHAML, and helping developers with their implementations, e.g., in JAVA, Octopus, Julia, Anasazi (Trilinos).
• proposing first preconditioned subspace iteration for eigenvalue computations, jointly with Dyakonov;
• developing Chebyshev methods for eigenvalue problems with applications to modelling criticality of nuclear reactors, jointly with Lebedev;
• defining and classifying preconditioned eigensolvers, separating the choice of a preconditioner from the choice of an iterative method;
• ?determining the optimal strategy for inner iterations of the inner-outer preconditioned Lanczos method;
• developing preconditioned domain decomposition eigensolvers with optimal performance, jointly with Skorokhodov;
• proposing multigrid preconditioning for eigensolvers, jointly with Neymeyr, with applications to image segmentation;
• deriving explicit sharp convergence rate bounds for many eigensolvers, some results jointly with Neymeyr et. al.;
• proposing soft locking deflation technique for eigensolvers that removes the locked approximate eigenvectors from iterations, but still updates them in the Rayleigh–Ritz method on the top of iterations;
• proposing a variational hybrid of the Rayleigh Quotient Iteration (RQI) that replaces the gradient direction with the RQI vector in the steepest descent method, leading to globally linear and asymptotically cubic convergence.

Ritz-Galerkin method, including Finite Element Method (FEM), is the main approximation technique for eigenvalue problems, also known as the Rayleigh–Ritz method. My research contributions included:

• connecting Lavrentiev regularization and Ritz approximation to derive FEM error estimates for linear differential equations with a jump in the leading coefficient, uniform in the value of the jump, jointly with?Widlund;
• deriving explicit error bounds for Ritz values approximating clustered eigenvalues, with applications to FEM for elliptic partial differential operators, jointly with Osborn;
• proving that the Ritz vectors are asymptotically the same as the projectors of the corresponding eigenvectors to the trial subspace in the Rayleigh–Ritz method;
• revealing a simple but deep connection of concepts of canonical correlations and the Rayleigh–Ritz method (namely that the squared cosines of the principle angles between subspaces X and Y are Ritz values corresponding to the subspace X for the orthogonal projector onto Y) and deriving pioneering majorization-type, including multiplicative, bounds on changes of the Ritz values when the subspace in the Rayleigh–Ritz method varies, with applications to FEM, jointly with my former PhD student Argentati;
• extending the Rayleigh–Ritz method to the case of infinitely dimensional projection subspace and bounding the Hausdorff distance between the Ritz values, corresponding to different trial subspaces, by a constant times the gap between the subspaces, jointly with my former PhD students Jujunashvili and Argentati;
• for self-adjoint possibly non-compact operators in Hilbert spaces, deriving accuracy bounds of the Rayleigh–Ritz method and bounds for the Rayleigh quotient and operator spectra;
• proposing and analyzing a polynomial version of the Temple-Lehmann method that bounds the eigenvalues from both above and below, in contrast to the Rayleigh–Ritz method that gives only bounds from inside;
• deriving sharp a priori eigenvalue error bounds of the Temple-Lehmann method, similar to those of the Rayleigh–Ritz method, and proving that the approximations to eigenvectors in Temple-Lehmann and Rayleigh–Ritz methods are asymptotically the same.

Ab initio calculations of material properties are based on first principles at atomic level. The most popular is the density functional theory (DFT). I collaborated with researchers from Département de Physique Théorique et Appliquée, CEA/DAM Ile-de-France and LRC – Centre de Mathématiques et de Leurs Applications, CNRS (UMR 8536) ENS Cachan to incorporate LOBPCG into DFT-based FORTRAN 90 parallel public code ABINIT in 2007, which still remains the default solver for parallel execution. For a single k-point, we obtained super-linear scaling for up to 100 processors and observed good performance up to 200 processors on distributed memory parallel computers. Langou and I adopted LOBPCG in PESCAN library for interior eigenvalues to simulate materials for solar panels. In a recent 2018 work with collaborators from Mitsubishi Electric, we proposed novel DFT+U type technology to determine the?band alignments?in quantum wells?using superlattice?calculations (with lattice relaxation), where the?valence and conduction band?offsets (VBOs and CBOs) were found from the projected?density of states?away from the interface. We applied this procedure to InGaAs/InP, InGaAs/InAlAs, and InAlAs/InP layers, and obtained both VBOs and CBOs consistent with experiments.

Model predictive control (MPC) is a topic of several of my patents for designing embedded devices to control in real time systems described by complex nonlinear dynamic models anticipating future events. MPC?is an increasingly popular in the industry method of?process control?satisfying constraints, e.g., via Active-Set?and?Interior Point?techniques. I co-invented preconditioned algorithms and utilizing mixed precisions for?MPC, resulting in?16 papers?and?several patent applications.?

Low-pass filters for graph-based signal processing (GB SP) modify a signal using the eigendecomposition of a matrix, commonly a normalized graph Laplacian, associated with a graph constructed from the signal. Most popular denoising filters can be viewed as graph-based operators, where the coefficients of the filter are interpreted as weights of a graph. I co-invented iterative GB SP algorithms, resulting in 7 papers?and?several patent applications, e.g., Chebyshev and conjugate gradient filters for?edge-preserving image denoising that project the image on a lower dimensional Krylov subspace of the graph Laplacian. I invented using negative graph weights for edge-enhancing signal denoising and presented at GlobalSIP 2015.

Guided Signal Reconstruction restores a signal?from a given sample using guiding operators, which increase signal components in the guiding set relative to those in a complementary set, e.g., iterative low-pass?edge-preserving filters?for?super-resolution?of images. Guiding is given by a low-pass filter and plays the role of?regularization, allowing controlling sample consistency and learning the filters, e.g., via?artificial neural networks, e.g.,?convolutional neural network.?Our guided iterative reconstruction is described in 5 papers and a patent.

Spectral co-clustering of DNA microarray gene expression data performs simultaneous clustering of genes and conditions from the data of synthesis of a functional?gene products, using partial SVD and PCA of data-related matrices. I contributed to this research topic jointly with my PhD students in the Department of Mathematical and Statistical Sciences, University of Colorado Denver in 2002-2009. We first implemented in MATLAB publicly available methods of Affymetrix (acquired?by?Thermo Fisher Scientific in 2016) GCOS software that perform single-array and comparison analysis. We improved the implementation of the MATLAB built-in probesetvalues function. Abram Jujunashvili in his PhD thesis used principle angles between subspaces to determine canonical correlations between groups of genes. While collaborating with and visiting the laboratory of Min Han at the Department of Molecular, Cellular, and Developmental Biology at the University of Colorado, I developed algorithms and MATLAB software for gene clustering, reported at the IACM/ECCOMAS Congress Venice, 2008 and Butcher Symposium 2009.

Spectral image segmentation partitions a?digital image?into segments for semantic segmentation, using?the eigendecomposition of a matrix, commonly a normalized graph Laplacian, associated with a graph constructed from image. I first applied and demonstrated parallel efficiency of LOBPCG with multigrid preconditioning for spectral image and video segmentation (jointly with my students), using our BLOPEX software. Popular skikit-learn library implemented my approach for spectral clustering using algebraic multigrid and SciPy LOBPCG version that Cimrman and I developed. I demonstrated that spectral image segmentation may be sensitive to down-sampling of the image due to entanglements of clustered eigenvalues and suggested performing the segmentation of the image at the original high resolution, due to the practically optimal linear complexity of the spectral image segmentation via LOBPCG with AMG preconditioning.

Spectral Clustering/Graph Partitioning performs unsupervised data clustering or finding patterns in the data by setting up a graph, determined by the data, formulating the clustering problem as graph partitioning, and calculating the partitions in the graph using the eigendecomposition of a matrix, commonly a graph Laplacian, associated with the graph. My contributions to spectral clustering (SC) included:

• inventing SC using negative spectrum of the graph Laplacian where graph weights can be negative, e.g., determined by correlations of the data points;
• co-inventing unsupervised anomaly detection in time series supported by partitioning variables representing dimensions of the time series data using SC;
• proposing arbitrary normalized graph Laplacian and analyzing the influence of normalization on spectral clustering, jointly with my PhD student by Donald McCuan.

Trap Lifetime Extraction analyzes experimental measurements to characterize electronic traps in semiconductors. I co-invented algorithms for estimating real lifetimes in logarithmic scale and corresponding coefficients from real-valued measurement data in logarithmic scale, where the data are multi-exponential, i.e. represented by linear combinations of decaying exponential functions with various lifetimes. The proposed method can also be used for nuclear magnetic resonance tomography and analyzing multi-lifetime components in fluorescence and other exponentially decaying processes.

Criticality of nuclear reactors describes the rate of the chain?fission reaction and needs to be computer simulated in real time to control the fission reaction in the reactor. Working at Kurchatov Institute, I developed algorithms and FORTRAN codes for criticality of neutron-physical reaction of water-cooled water-moderated energy pressurised water reactor WWER-1000; see V.S. Apokorina, A.V. Knyazev, "BIPR-5M software code", 1986, VANT, Physics and Technology of Nuclear Reactors: Physics and Computational Methods of Nuclear Reactors, 5, 34-35.?