Graph of Thoughts with LLMs; GPT Can Solve Math Problems; Bias and Fairness in LLMs; Ensembling Techniques – Weekly Concept; and More.

Editor's Paper Recommendations

Graph of Thoughts: Solving Elaborate Problems with Large Language Models: The paper introduces Graph of Thoughts (GoT). This framework advances prompting capabilities in large language models (LLMs) beyond those offered by paradigms such as Chain-of-Thought or Tree of Thoughts (ToT). The key idea and primary advantage of GoT is the ability to model the information generated by an LLM as an arbitrary graph, where units of information (“LLM thoughts”) are vertices, and edges correspond to dependencies between these vertices. This approach enables combining arbitrary LLM thoughts into synergistic outcomes, distilling the essence of whole networks of thoughts, or enhancing thoughts using feedback loops. We illustrate that GoT offers advantages over the state-of-the-art on different tasks, for example, increasing the quality of sorting by 62% over ToT while reducing costs by >31%. The study ensures that GoT is extensible with new thought transformations and thus can be used to spearhead new prompting schemes. This work brings LLM reasoning closer to human thinking or brain mechanisms such as recurrence, which form complex networks.

GPT Can Solve Mathematical Problems Without a Calculator: Previous studies have typically assumed that large language models can only accurately perform arithmetic operations, particularly multiplication of >8 digits and operations involving decimals and fractions, using calculator tools. This paper aims to challenge this misconception. With sufficient training data, a 2 billion-parameter language model can accurately perform multi-digit arithmetic operations with almost 100% accuracy without data leakage, significantly surpassing GPT-4 (whose multi-digit multiplication accuracy is only 4.3%). We also demonstrate that our MathGLM, fine-tuned from GLM-10B on a dataset with additional multi-step arithmetic operations and math problems described in the text, performs similarly to GPT-4 on a 5,000-samples Chinese math problem test set.

Bias and Fairness in Large Language Models: A Survey: Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere. Despite this success, these models can learn, perpetuate, and amplify harmful social biases. This paper presents a comprehensive survey of bias evaluation and mitigation techniques for LLMs. We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing, defining distinct facets of harm and introducing several desiderata to operationalize fairness for LLMs. We then unify the literature by proposing three intuitive taxonomies: two for bias evaluation, namely metrics and datasets, and one for mitigation. Our first taxonomy of metrics for bias evaluation disambiguates the relationship between metrics and evaluation datasets. It organizes metrics by the different levels they operate in a model: embeddings, probabilities, and generated text. Our second taxonomy of datasets for bias evaluation categorizes datasets by their structure as counterfactual inputs or prompts and identifies the targeted harms and social groups; we also release a consolidation of publicly available datasets for improved access. Our third taxonomy of techniques for bias mitigation classifies methods by their intervention during pre-processing, in-training, intra-processing, and post-processing, with granular subcategories that elucidate research trends. Finally, we identify open problems and challenges for future work. Synthesizing a wide range of recent research, we aim to provide a clear guide of the existing literature that empowers researchers and practitioners to understand better and prevent bias propagation in LLMs.

--

Are you looking to advertise a product, job opening, or event to an audience of over 35,000 AI researchers and engineers? Get in touch with us on?LinkedIn?to explore your options.

Enjoy the newsletter? Help us make it bigger and better by sharing it with colleagues and friends.

--

Weekly Concept Breakdown

In the analytical framework of machine learning, ensemble techniques hold a significant position, enhancing predictive accuracy and robustness by integrating multiple models. Let’s overview a few ensemble techniques:

Gradient Boosting: Utilizing gradient descent, a cornerstone in optimization algorithms, it iteratively minimizes the error of the predictive model. This sequential correction of predecessor models' errors facilitates refined predictive accuracy, a feature that has made it a stalwart in data science competitions and complex predictive modeling tasks.

Bayesian Ensemble Optimal Classifier: This technique stands at the intersection of Bayesian principles and ensemble strategies, orchestrating a collaborative effort of multiple models to derive the most optimal prediction. It leverages probabilistic theories to assess uncertainties accurately, offering a scientifically rigorous approach to artificial intelligence applications.

Stacked Generalization (Stacking): Stacking involves training a meta-model to combine the predictions of multiple base models effectively. This hierarchical approach allows for a nuanced integration of various models, capturing intricate patterns and relationships in the data that individual models might miss.

Strengths of Ensemble Techniques

Aggregating diverse models often achieves higher accuracy than individual models.

Ensemble methods are less likely to overfit, ensuring a more stable and reliable model.

Drawbacks

Integrating multiple models can lead to a complex system, making it challenging to interpret and analyze.

These techniques can be computationally intensive, requiring substantial resources and time.

Applications

Finance: Ensemble techniques aid in detecting fraudulent transactions, which are typically rare events, by identifying patterns that deviate significantly from normal transactions.

Healthcare: In medical imaging, these techniques can help identify rare pathological findings, enhancing the accuracy of diagnoses by flagging anomalous patterns in medical images.

Cybersecurity: Ensemble learning is pivotal in identifying security breaches by detecting unusual patterns in network traffic, thereby enhancing the security of systems.

Manufacturing: In quality control processes, ensemble techniques can detect defective products by identifying anomalies in the production data.

Growth Zone

• Become More Comfortable Making Bold Decisions
• How to Stop Overthinking and Start Trusting Your Gut

Expert Advice