??Top ML Papers of the Week

The top ML Papers of the Week (Oct 30 - Nov 5):

1). MetNet-3 - a state-of-the-art neural weather model that extends both the lead time range and the variables that an observation-based model can predict well; learns from both dense and sparse data sensors and makes predictions up to 24 hours ahead for precipitation, wind, temperature, and dew point. (paper | tweet )

2). Evaluating LLMs - a comprehensive survey (100+ pages) on evaluating LLMs, including discussions about the different types of evaluations, datasets, techniques, and more. (paper | tweet )

3). Battle of the Backbones - a large benchmarking framework for a diverse suite of computer vision tasks; find that while vision transformers (ViTs) and self-supervised learning (SSL) are increasingly popular, convolutional neural networks pretrained in a supervised fashion on large training sets perform best on most tasks. (paper | tweet )

4). LLMs for Chip Design - proposes using LLMs for industrial chip design by leveraging domain adaptation techniques; evaluates different applications for chip design such as assistant chatbot, electronic design automation, and bug summarization; domain adaptation significantly improves performance over general-purpose models on a variety of design tasks; using a domain-adapted LLM for RAG further improves answer quality. (paper | tweet )

5). Efficient Context Window Extension of LLMs - proposes a compute-efficient method for efficiently extending the context window of LLMs beyond what it was pretrained on; extrapolates beyond the limited context of a fine-tuning dataset and models have been reproduced up to 128K context length. (paper | tweet )

Sponsor message

DAIR.AI presents a new cohort-based course, Prompt Engineering for LLMs , that teaches how to effectively use the latest prompt engineering techniques and tools to improve the capabilities, performance, and reliability of LLMs. Enroll here . Feel free to use this 20% discount promo code: MAVENAI20

6). Open DAC 2023 - introduces a dataset consisting of more than 38M density functional theory (DFT) calculations on more than 8,800 MOF materials containing adsorbed CO2 and/or H2O; properties for DAC are identified directly in the dataset; also trains state-of-the-art ML models with the dataset to approximate calculations at the DFT level; can lead to important baseline for future efforts to identify MOFs for a wide range of applications, including DAC. (paper | tweet )

7). Symmetry in Machine Learning - presents a unified and methodological framework to enforce, discover, and promote symmetry in machine learning; also discusses how these ideas can be applied to ML models such as multilayer perceptions and basis function regression. (paper | tweet )

8). Next Generation AlphaFold - reports progress on a new iteration of AlphaFold that greatly expands its range of applicability; shows capabilities of joint structure prediction of complexes including proteins, nucleic acids, small molecules, ions, and modified residue; demonstrates greater accuracy on protein-nucleic acid interactions than specialists predictors. (paper | tweet )

9). Enhancing LLMs by Emotion Stimuli - explores the ability of LLMs to understand emotional stimuli; conducts automatic experiments on 45 tasks using various LLMs, including Flan-T5-Large, Vicuna, Llama 2, BLOOM, ChatGPT, and GPT-4; the tasks span deterministic and generative applications that represent comprehensive evaluation scenarios; experimental results show that LLMs have a grasp of emotional intelligence. (paper | tweet )

10). FP8-LM - finds that when training FP8 LLMs most variables, such as gradients and optimizer states, in LLM training, can employ low-precision data formats without compromising model accuracy and requiring no changes to hyper-parameter. (paper | tweet )