Which Generative AI for Data Analysis? Insights from 10 AIs Performing a Multiple Linear Regression.

During my teachings on AI-assisted data analysis, the question inevitably arises as to which AI to recommend to students for data analysis. Here is a personal benchmark using the same regression analysis (prompt below). For each AI evaluated, I assessed whether it was capable of accurately interpreting the three regression tables based on relevant information from the tables. Here is what I can convey to the students (I should note that for researchers, these tools are too basic to foster meaningful discussions and that only a "fine tuned" AI can facilitate a "customized" academic discussion).

AIs tested (all tested AIs are free versions to ease access for students): ChatGPT 3.5 (Open AI), Gemini (Google), Mixtral 8x7b (Mistral AI), Claude3 Sonnet (Anthropic AI), Pi (Inflection AI), Llama 2.70b (Meta), Sonar medium (Perplexity AI), Llava 1.6 (University of Wisconsin, Columbia, Microsoft), Gemma 2b (Google), Gemma 7b (Google).

Firstly, the good news is that, for most of the AIs tested, the analysis of regression tables is accurate. The AIs are capable of identifying the coefficient of determination (R2), linking it to the ANOVA test (entire model), and finally providing relevant analysis for each parameter (Table 3). However, it should be noted that there was a reading error of Table 3 by Llama 2.70B (Meta), but nonetheless, a good interpretation of what was read. Llava 1.6 is somewhat weak in interpreting tables. It does not make mistakes, but it really does not provide enough information (as a reminder, this AI was designed more for images than for data analysis). Lastly, the lighter version of Gemma (Google), the 2B, makes interpretation errors. If I were to offer advice, the AIs that perform best at this stage are Gemini (Google), Pi (Inflection AI), Mixtral 8x7b (Mistral AI), and Claude3 Sonnet (Anthropic AI).

In the prompt (see at the end of this article), I requested a specific formatting for the statistical results. Most AIs complied with this request except for Gemma2B, as it did not reproduce the figures from the tables as asked. Mixtral 8x7B did not follow the exact format but provided a very close and formally correct format.

I also observed whether the "scientific" formatting of the results was extended to other results beyond those for which formatting was requested. I had only given the example of F. It was interesting to see if the AI respected this formatting for the parameters in Table 3, even though I had not requested it. Overall, the format is respected everywhere except for Sonar Medium (Perplexity AI), Llava 1.6, and Gemma 2B and 7B (Google).

While I had not requested it, three AIs suggested academic references to illuminate the results: Gemini (3 references), Mixtral 8x7b (2 references), and Pi (2 references). However, only two of these AIs provide references that actually exist. Indeed, Mixtral suggests articles that do not exist. For Gemini and Pi, the references exist and are quite relevant, even if somewhat dated (the model tested is also not based on recent hypotheses).

I then noted whether the response was structured in a way to formally present a title for statistical, theoretical, and managerial comments. The three Google AIs (Gemini, Gemma 2b, Gemma 7b) format their responses very clearly. This is also the case for Pi (Inflection AI).

Finally, I examined the number of characters dedicated to statistical, theoretical, and managerial comments. Pi (Inflection AI) significantly stands out with its very detailed comments, followed by Mixtral 8x7b, Claude3 Sonnet, and Gemini free (Google).

In conclusion, compared to the paid version of ChatGPT (4 turbo), which performs very well, I have noticed that some free AIs can compete with or even surpass ChatGPT4 for this regression analysis. Personally, I find that the AIs Pi, Mixtral, Claude3 Sonnet, and Gemini provide a comparable or even better analysis for free, and in all cases, much better than ChatGPT 3.5 (free version). Additionally, I encountered some issues with ChatGPT 3.5, which in several attempts refused to conduct the analysis for various reasons.

Therefore, when choosing a free version for conducting data analyses, I would recommend to students to use Pi, Mixtral (with caution to verify the bibliographic references it provides), Claude3 Sonnet, or Gemini. An advantage of these AIs is that they also have chatbots directly accessible on their websites (unlike Gemma or Llama, for example).

#DataAnalysis #AI #ArtificialIntelligence #EducationalTechnology #DataScience #TeachingAI #FreeAI #Chatbots

The prompt :

Here are the results of a multiple linear regression analysis. These results, in tabular form, were obtained using SPSS software. In this regression, the dependent variable is purchase intention (pi), and the two explanatory variables are attitude towards the ad (aad) and attitude towards the brand (ab). The model also includes a constant. The three variables (pi, aad, ab) are measured on a Likert scale ranging from 1 to 7. A score of 1 indicates a very low attitude or intention. Conversely, a score of 7 indicates a strong attitude or intention. The underlying hypotheses of this research posits that attitude towards the ad and attitude towards the brand significantly influence purchase intention.

There are three tables summarizing the results.

Here is Table 1 - Model summary (separator: tabulation) :

Model R R-squared Adjusted R-squared Standard error of estimate

1 ,334 ,111 ,108 1,731

Here is Table 2 - ANOVA (separator: tabulation):

Model 1 Sum of squares ddl Mean square F Sig.

Regression 183,309 2 91,655 30,574 <,001

Residuals 1462,927 488 2,998

Total 1646,236 490

Here is Table 3 - Coefficients (separator: tabulation):

Model Non-standardized coefficients Standardized coefficients

1 B Standard error Beta t Sig.

(Constant) ,191 ,354 ,538 ,591

aad ,249 ,052 ,206 4,762 <,001

ab ,341 ,064 ,231 5,335 <,001

Analyze and comment on the results of this regression. You will write in an academic, formal style suitable for an audience of ph.d in management. You will write in an editorial style, avoiding bulleted lists, and referring in the text to the results of the analysis by quoting them in a formal style (for example: F[ddl1;ddl2] = value, p<.005). You will first analyze the results from a statistical point of view, followed by a theoretical commentary, and conclude with managerial implications.