Machine Learning vs. XAFS Interpretation

In an earlier post, I discussed the access problem for x-ray absorption fine structure (XAFS) and how benchtop spectrometers have the potential to enable a wide range of 'routine' analytical studies that fit poorly with the necessary use and allocations models for synchrotron beamtime. However, raw instrument availability is only one of the steps that will be needed for broader use of this technique. None of us, in our hearts, care about the spectra themselves -- we care about their interpretation.

For this reason, it is interesting that there are early signs of an emerging battle of titans that strikes at the heart of XAFS interpretation: the unstoppable force of machine learning is being aimed at the seemingly immovable object of the XAFS inverse problem. The outcome could have dramatic impact on the future range of application of XAFS, both by enabling new classes of studies by experts and also by creating new classes of analytical applications where routine analysis can be custom-trained and automated. The latter development would greatly lower the bar of individual expertise needed for first use of XAFS, perhaps especially in industry.

The forward problem for XAFS, illustrated on the left panel of the figure, can be stated as: Given the structure of a system, calculate the resulting spectrum. There is an extensive body of work on the forward problem, beginning with the 'EXAFS equation' from the seminal Sayers, Stern and Lytle paper in 1971 and continuing through stages such as the several decade effort of the FEFF project for scattering phase factors, full-multiple scattering treatments to better address the near edge structure (XANES), charge-transfer multiplet approaches, and k-space treatments using the Bethe-Salpeter equation.

At the same time, the difficulty of the inverse problem, illustrated in the right panel of the figure, has been a constant topic of discussion and research. For convenience, I’ll broadly define the inverse problem to encompass not only the ‘true inverse’ task of identifying a pure substance being measured but also the more general (and typically more useful) inference of physical observables from measured spectra on more disordered systems. Oxidation state, coordination numbers, coordination symmetry, ligand speciation, and bond length are common observables, and they form our ‘classifications’ in the terminology of machine language. 

The point of using machine learning (ML) approaches for XAFS is that the strong status of theory for the forward problem provides the basis for training a neural network to address the inverse problem.

Here, I'll briefly summarize two recent papers that bring machine learning approaches to bear on the XAFS inverse problem. The first work focuses the information content in the XANES of a narrow class of catalysts that have seen very heavy study, and hence for which there is strong prior information that helps to constrain the training of the neural network. The second is a broader, materials-genome study that takes an important step toward understanding what general level of expert system performance is possible in the extreme case when no system-specific training is given. The fact that these papers inhabit the two distant endpoints of prior knowledge constraining the directed ML training is particularly helpful for imagining the scope of future directions.

Supervised Machine Learning Based Determination of Three-Dimensional Structure of Metallic Nanoparticles (Timoshenko, et al., 2017)

This recent work, Timoshenko et al, by a collaboration of researchers at Stony Brook University and Brookhaven National Labs is very interesting. Those authors focus on the problem of identifying the cluster content and geometry for Pt nanoclusters having strong catalytic activity on an Al2O3 support. This system has seen a large number of extended XAFS (EXAFS) studies and, while successful, those studies have struggled with low information content because of thermal disorder at the high temperatures that are needed for the catalytic activity.

This raises the question of instead using the XANES, that is the fine structure that is close to the absorption edge. There is a key benefit in this choice. The XANES is much less degraded by thermal disorder than is the EXAFS, this is due to the longer deBroglie wavelength of lower energy photoelectrons. However, the decision to analyze the XANES also comes with a drawback: there isn't a first, model-independent step that can be used to interpret spectra to get at the desired atomic-scale descriptors. For EXAFS, Fourier transform analysis gives a first such descriptor for bond lengths and coordination, although corrections for phase shifts are needed for greater accuracy. When using only the XANES, researchers must instead use some degree of trial-and-error based on their experience, intuition, or the results of MD-DFT structural simulations as input to first-principles calculation. 

This is where machine learning tools can play a novel role. Since the forward problem is reasonably well solved for Pt clusters using existing theoretical tools, it is possible to create an ensemble of simulated spectra with known classifications (oxidation state, etc.) as the basis for training a neural network. In this present case, Timoshenko, et al., created synthetic Pt L3 XANES spectra for a broad ensemble of feasible Pt nanoclusters having different numbers of atoms and cluster packing geometries, and then used those spectra to train a simulated neural network.

While full details are in the Timoshenko, et al., manuscript, a conceptual schematic of the general training routine is shown in the left panel of the figure below.  

Using physical intuition about the specific problem, an ensemble of meaningful Pt nanoclusters is defined. Next, a clever combinatorial approach based on unique absorber positions in the model clusters is used to accelerate the calculation of several hundred thousand synthetic XANES spectra by the FEFF code, and each such spectrum is assigned effective values for the site-averaged coordination numbers, C1, C2, C3 and C4. This linked-pair of input spectrum and classifications (the coordination numbers) is then used to train a neural network. The training is continued over the ensemble of Pt nanoclusters until the internal degrees of freedom of the neural network converge and training is complete.

The trained neural network can then be used to interpret experimental XANES spectra, as shown in the right panel of the figure. For each experimental spectrum the neural network infers the four coordination numbers, from which the cluster geometry is then deduced by the user. The results in Timoshenko, et al., hold two points that are worthy of specific notice. First, the accuracy of the ML-inferred coordination numbers from just the XANES is quite good when compared to the values inferred (by humans) from the EXAFS. This is important: it is far easier and faster to measure high-quality XANES than high-quality EXAFS. Second, it is true that the need for full-multiple scattering treatment in the XANES region makes it clear that this fine structure contains, in principle, a plethora of local and intermediate-range structural information. Humans have known this fact. However, the ML-trained neural network found information in the XANES that we, mere humans, haven’t previously found. Score one for the machines.

Automated Generation and Ensemble-Learned Matching of X-ray Absorption Spectra (Zheng, et al, 2017)

The second paper, Zheng, et al., comes from a collaboration of researchers at UC San Diego, Lawrence Berkeley National Lab, UC Berkeley, the University of Washington, SUNY Binghamton, and the Center for Disease Control. Unlike the prior study, which benefitted from strong prior information about the system's chemistry and coordination, these authors instead ask a typical expert-system question: Can we train a computer to reliably interpret XAFS spectra for a very wide range of problems?

This study again uses a machine learning methodology. Focusing on the XANES, 300,000 synthetic spectra (from the theory codes for the forward problem) were calculated for materials in an open source database associated with the Materials Project database. A neural network was then trained on the resulting linked-pairs of synthetic spectra and classification of the material identity, oxidation state, and coordination.

The results are again very interesting. With no prior information to aid interpretation of randomly-selected experimental spectra, the Ensemble-Learned Spectra IdEntification (ELSIE) algorithm infers each of the oxidation state and coordination with approximately 80% accuracy. It also achieves about 60% accuracy for a weak measure of material identification, i.e., whether any of the five ‘most similar’ theory reference spectra are actually the substance measured by the experiment. 

There is a, however, a broad question that the Zheng, et al., paper (perhaps too kindly) delays for the moment: How would the ‘typical’ expert human users of non-machine-learning aided standard XAFS analysis tools match up against the performance of ELSIE?  Perhaps humans still have an edge for this much broader problem...

The future for Machine Learning interpretation of XAFS spectra

Predicting the future is never easy, but at least two directions seem worth discussion.

First, we should wonder how the community of (human) experts will find new uses for ML-driven analytics of XAFS. There could be considerable low-hanging fruit, especially if it turns out to be generic that such complete information about coordination is subtly encoded into the XANES. From a more brute-force perspective, it is also very promising to note that ML significantly encompasses linear mathematics (such as least square fit routines) and also probabilistic issues (such as, with some debate about terminology, maximum likelihood approaches). This raises the question of whether much of the human-time-intensive work to interpret EXAFS spectra can be trained into a neural network. It feels unlikely that EXAFS analysis is, in an information theoretical sense perhaps, as difficult as recognizing human speech, visualizing and interpreting text, or beating the world’s most skilled humans at the game of Go, to name only a few problems where ML approaches have thrived.

Second, from the perspective of both education and also, admittedly, benchtop XAFS commercialization, it is interesting to consider how ML-based expertise can bring XAFS to the coupled system of completely new applications and completely new users, especially de novo users. Educational and industrial users of x-ray powder diffraction, to name a close comparison case, are seldom high-level experts on Reitveld analysis, nor do many of their applications usually need world-class XRD analytical tools. Instead, user-friendly ‘intelligent enough’ software packages allow a fast first-level analysis for the obvious classifications, such as the crystal structure, possible material identities, and domain sizes. The very early steps for ML applied to XAFS give hope for some degree of generalist expert system. They give even stronger hope for specialized problems, such as would greatly aid the adoption of XAFS in industrial quality control where significant prior information would be available.