Bard/Mathpix qualitative comparison - Converting images of Cosmological equations to LaTex

Introduction

Lior Alexander 's helpful post the other day taught me that Bard now apparently has the capacity to transcribe a mathematical equation from an image into the corresponding LaTex form. My PhD thesis on Galaxy Clusters was written entirely in LaTex, and honestly, it wasn't the most fun software to use, especially when writing more involved Cosmology-related equations. So I wanted to test out if Bard would have indeed helped speed up my thesis writing with its new functionality if I had had access to it while writing my dissertation. My initial findings weren't too encouraging (read that post here) but I wanted to delve a little deeper, hence this article. Valeriy Manokhin, PhD, MBA, CQF suggested I also try out Mathpix, so I have uploaded the same image of 3 very different (but important!) equations into Bard and Mathpix, and provided side-by-side comparisons of the respective results in Overleaf.

I hope the article imbues in the reader a sense of Cosmological wonder, and a gentle wariness about harnessing the powers of Generative AI without deeper investigation. Let's get into it...

Comparisons

1. Comoving Distance

Comoving distance is the distance between two points measured along a path defined at the present?cosmological time.?It is the distance between observers that are both moving with the Hubble flow, and it does not change with time, as this distance accounts for the expansion of the universe. The comoving distance from an observer to a distant object can be computed by the following formula:

where?a(t′) is the?scale factor,?t_e (Interesting that I couldn't add a subscript in this LinkedIn post)?is the time of emission of the photons detected by the observer,?t?is the present time, and?c?is the?speed of light?in vacuum. Note that since the scale factor a(t) = 1/(1+z) (where z is the redshift) the above equation reduces to a Euclidean form at low redshifts.

I uploaded the image of this equation in both Bard (with the prompt "Please convert the following equation to LaTex") and Mathpix.

The LaTex code returned by Bard is below:

t \int_0^t a(t') x \,dt' = \chi \cdot c \cdot e^t        

... and this is the code provided by Mathpix:

\chi=\int_{t_e}^t c \frac{\mathrmzj3nl9r5 t^{\prime}}{a\left(t^{\prime}\right)}        

Now, we use the Overleaf editor to convert each of the LaTex snippets to mathematical equations that would be used in the pdf version of a research paper. And the results are below (Please note that Equation (1) is from Bard and Equation (2) is from Mathpix):

As we can see, Bard does some strange things: What is happening to t_e? Where are we getting the exponential? However, Mathpix is exactly correct in its output. Let's move on...

2. Einstein Field Equations

The Einstein Field Equations describe gravity as a result of spacetime being curved by mass and energy and are a core part of General Relativity. These equations are based on the idea that space and time can be considered part of the same fabric and that the curvature of spacetime tells both matter and energy how to move within it. The basic idea is beautiful "matter and energy tells spacetime how to curve, and that curved spacetime, in turn, tells matter and energy how to move" and entirely revolutionary (Read more here).

The Field Equations are shown below:

while the image below explains the terms:

I uploaded the image of this equation to both Bard and Mathpix.

The LaTex code returned by Bard is below:

8πG - Tuμνμν = 0        

This isn't looking promising already.... but let's also take a look at the Mathpix returned LaTex code before drawing any conclusions:

G_{\mu \nu}+g_{\mu \nu} \Lambda=\frac{8 \pi G}{c^4} T_{\mu \nu}        

We use the Overleaf editor to convert each of the LaTex snippets to mathematical equations that would be used in the pdf version of a research paper. And the results are below (Please note that Equation (1) is from Bard and Equation (2) is from Mathpix):

Once again, Mathpix reproduces the equation exactly, while Bard definitively does not do a good job. There does seem to be a theme here, but let us also try it out on a 3rd Cosmological equation...

3. The Press-Schechter Formalism

For the last example, we take a mathematical model that predicts the number of objects (such as galaxy clusters) of a certain mass in a given volume of the universe. For cold dark matter cosmological models, perturbations on all scales are imprinted on the universe at very early times, and over large time-scales, these perturbations collapse and form objects such as galaxies or galaxy clusters via hierarchical structure formation. This formalism is crucial in trying to predict the abundance of galaxy clusters.

This equation is shown below (depicting the number of objects between masses M and M + dM):

where?n?is the index of the power spectrum of the fluctuations in the early universe, the density term rho (LaTex typesetting would have been greatly helpful here!)?is the mean (baryonic and dark) matter density of the universe at the time the fluctuation from which the object was formed had gravitationally collapsed, and?M??is a cut-off mass below which structures will form.

Uploading the image of the above equation into Bard and Mathpix results in the following code. First for Bard:

dn = N(M) dM \left( 1 + \frac{M(3+)}{6M(M+3)/3} \right) e^{dM}        

And for Mathpix:

d n \equiv N(M) d M=\frac{1}{\sqrt{\pi}}\left(1+\frac{n}{3}\right) \frac{\bar{\rho}}{M^2}\left(\frac{M}{M^*}\right)^{(3+n) / 6} \exp \left(-\left(\frac{M}{M^*}\right)^{(3+n) / 3}\right) d M        

The Bard version is much shorter than the Mathpix version, which, if experience with writing formulae in LaTex has taught me, is probably not a good sign. But let's see what we get after we use the Overleaf editor to convert each of the LaTex snippets to mathematical equations that would be used in the pdf version of a research paper.

And the results are shown below (Please note that Equation (1) is from Bard and Equation (2) is from Mathpix):

Bard is unable to even transcribe an equivalence successfully, and adds in elements which do not make sense (M(3+)). Mathpix reproduces the original equation exactly.

Moving on to the finale...

Conclusions

In this article, we tested out the LaTex transcribing capabilities of both BARD and Mathpix when presented with the image of an equation. The LaTex code generated by the two programs were then compiled into a pdf-readable form using Overleaf. Three Cosmology-related equations were used for the demonstration, the Comoving distance, the Einstein Field Equations and the Press-Schechter Formalism. In all 3 cases, Bard failed to reproduce the original equation while Mathpix succeeded exactly.

It would seem that at this stage, the OCR capabilities of Bard, at least regarding mathematical formulae, should not be relied upon, while Mathpix seems like it is very capable in this regard, These are anecdotal opinions, but I have tried several other examples as well, and, for the current version of Bard, the results are never more encouraging.

It would also be interesting to see what the LaTex output would be if one were to provide Bard with text prompts for the more famous equations. But that's for another article, and possibly a comparison with other LLM-based chat interfaces.

To summarize, at this stage, it seems that Bard would absolutely not have sped up my workflow if I were still working on writing equations in LaTex for my PhD thesis. But perhaps in future iterations...

References

1. Dodelson - Modern Cosmology
2. Peacock - Cosmological Physics
3. Binney and Tremaine - Galactic Dynamics