#14 AI Research News Updates

Hey Folks,

Thank you so much for signing up for my AI Newsletter. In the last few days, we were doing deep research in AI-related research updates and we were able to find these cool ones.

But before you start reading, please?Join Our Subreddit?so you don't miss any updates.

?? Researchers from Carnegie Mellon University recently published a paper that compares existing code models – Codex, GPT-J, GPT-Neo, GPT-NeoX, and CodeParrot – across programming languages. By comparing and contrasting various models, they want to offer more light on the landscape of code modeling design decisions, as well as fill in a major gap: no big open-source language model has been trained purely on code from several programming languages. Under the umbrella name?“PolyCoder,”?the team proposes three such models with parameters ranging from 160M to 2.7B.

?? ByteDance researchers introduce?MetaFormer, which combines vision with meta-information using a transformer. With the help of meta-information, MetaFormer may effectively increase the accuracy of FGVC. MetaFormer can alternatively be thought of as a hybrid structural backbone, with the convolution downsampling the image and introducing the convolution’s inductive bias and the transformer fusing visual and meta-information. MetaFormer also provides a robust baseline for FGVC without the bells and whistles in this way.

?? Northwestern University researchers present a new design that combines a traditional CPU with a systolic convolutional neural network (CNN) accelerator on a single core, resulting in a highly programmable and versatile unified design. The study team says it can achieve a core utilization rate of above 95%. Data transfer is removed, and latency for end-to-end ML operations is minimized using this method.

?? A new study by researchers in Italy and Jordan studied the voice of Parkinson’s disease patients in a large and clinically well-characterized cohort. This study is the first to classify voice in Parkinson’s disease patients based on the stage and severity of the disease and the effect of chronic L-Dopa medication. All diagnostic tests were evaluated for sensitivity, specificity, positive and negative predictive values, and accuracy.

?? A group of Stanford University scientists employed autonomous drone technology and a machine learning approach to focus their efforts on discovering and gathering the most valuable data in Antarctica to increase our understanding of the processes that drive sea-level rise.?

?? Researchers from the University of Washington and UC San Diego introduce Tensor Query Processor (TQP) with Tensor Computation Runtimes for Query Processing – 20x speedup. TQP is the first query processor to operate on TCRs, according to the researchers, and it has been shown to enhance query execution speed by up to 20 times over CPU-only systems and up to 5 times over specialist GPU solutions.

?? Researchers from Carnegie Mellon University identified a stronger observation first in a recent investigation. Consider two neural networks that were trained with the same hyperparameters and dataset but with different random seeds (for example, the data may be given in various random orders, and/or the network weights could be randomly initialized). Given that both models observe the same data, one would expect the percentage of disagreement to be substantially lower than in previous experiments.?However, the team finds that the disagreement rate is still roughly equivalent to the test error on the SVHN and CIFAR-10/100 datasets, as well as for variations of Residual Networks and Convolutional Networks, only slightly diverging from previous studies.

?? Amazon uses Machine Learning to improve video quality on Prime Video. To validate new program releases or offline modifications to encoding profiles, Prime Video’s Video Quality Analysis (VQA) division began employing machine learning three years ago to discover faults in collected footage from devices such as consoles, TVs, and set-top boxes. More recently, Amazon has used the same techniques to solve problems like real-time quality monitoring of our thousands of channels and live events, as well as large-scale content analysis.

?? Microsoft’s latest Machine Learning research introduces μTransfer: a new technique that can tune the 6.7 billion parameter GPT-3 model using only 7% of the pretraining compute. Their scaling theory allows the creation of a method for transferring training hyperparameters across model sizes. If μP networks of different widths have comparable training dynamics, they will likely have similar optimal hyperparameters. As a result, they should simply apply the best hyperparameters from a tiny model to a larger version. However, the findings show that P can accomplish the same result with no alternative initialization and learning rate scaling rule. This practical method is referred to as “μTransfer.”

?? The University of Surrey’s Centre for Vision, Speech and Signal Processing and the BBC are collaborating to test a mechanism to utilize AI’s progress in media customization called Artificial Intelligence for Personalised Media Experiences, or?AI4ME. Researchers are experimenting with “citizen councils” to create a conversation in which the councils’ comments will inform the development of technology. Their citizen council could have a lot of examples and be independent of the BBC.

?? A new Google research has implemented a production ML model that employs federated learning with a stringent differential privacy guarantee, following a multi-year, multi-team effort spanning foundational research and product integration. The team used the DP-FTRL approach to train a recurrent neural network to power next-word prediction for Spanish-language Gboard users during deployment. This neural network is trained directly on user data with a formal DP guarantee. Furthermore, the federated approach provides further data minimization benefits, and the DP guarantee safeguards all data on each device, not just individual training samples.

?? Researchers from the University of Hamburg propose a machine learning model, called ‘LipSound2’, that directly predicts speech representations from raw pixels. The purpose of the paper presented in this article is to reconstruct speech only based on sequences of images of talking people. The generation of speech from silent videos can be used for many applications: for instance, silent visual input methods used in public environments for privacy protection or understanding speech in surveillance videos.

?? A team from LinkedIn and CMU devised a novel GNN method called “Performance-Adaptive Sampling Strategy,” or “PASS,” and open source?the implementation of PASS?that selects appropriate neighbors using an AI algorithm. The AI model developed by PASS learns how to choose neighbors who improve the predicted accuracy of the GNN model. By examining the attributes of a given neighborhood, the AI model determines whether or not to select that neighbor. This method has the advantage of working well independent of the job for which the GNN model is being employed.

?? Google researchers propose a new method, called?Constrained Instance reWeighting (CIW), that dynamically assigns importance weights to individual instances and class labels in a mini-batch, to reduce the effect of potentially noisy examples. The researchers define a family of restricted optimization problems that offer easy solutions for these crucial weights. These optimization issues are tackled in mini-batch increments, eliminating the need to keep and update significant weights throughout the dataset.

Before you leave this page, please?Join Our Subreddit?so you don't miss any updates.

Feel free to share your research update/ startup news with us at [email protected]        

About:?

Asif Razzaq:?Asif Razzaq is an AI Journalist and Cofounder of Marktechpost, LLC. He is a visionary, entrepreneur, and engineer who aspires to use the power of Artificial Intelligence for good.

Email:[email protected]