How can machine learning be used to improve existing algorithms?
Machine Learning
Perspectives from experts on the questions that matter for Machine Learning
This article was an early beta test. See all-new collaborative articles about Machine Learning to get expert insights and join the conversation.
Algorithms already play a major role in industries that require processing and analyzing large sets of data. But machine learning can take things one step further, making algorithms that much stronger and more useful. Here are some ways that machine learning can improve algorithms that already exist.?
1. Tuning parameters: One of the challenges of using algorithms is selecting the right parameters . For example, the learning rate is a parameter that controls how much an algorithm learns from each data point. If the learning rate is too high, the algorithm may overfit the data, fitting exactly against the data it is trained on and being unable to perform accurately against unseen data. If the learning rate is too low, the algorithm may not learn enough from the data. Machine learning can be used to automatically tune the learning rate so that users will be able to glean the most useful analytics.?
2. Increasing flexibility: Traditional algorithms are often brittle, meaning they perform well on the data they were trained on but break down when faced with new data. Machine learning algorithms, on the other hand, are more robust and can handle changes in data better. They can also generalize better than traditional algorithms, since they are able to capture subtle patterns.?
“I think we're entering a new era of automation. Since [machine learning] algorithms can learn, this opens up the possibility of flexible, adaptable automation.”
— Kence Anderson is the director of autonomous AI adoption at Microsoft. He holds over 18 years of experience in the technology industry.
领英推荐
3. Creating new designs: At times, designing a new algorithm can be more effective than trying to adapt existing ones to your needs. For example, with so many different ways to design a neural network, users may end up choosing a less than ideal algorithm for their goals. Machine learning, however, can improve the process by automatically searching for the best neural network design for a given task.?
“Selecting the most relevant features to include in your model not only helps you reduce overfitting, it also helps your model run faster.”
— Ngwa Bandolo Bobga Cyril is a senior manager of data analytics at telecommunications company YouMee Mobile. He holds over 12 years of experience in the tech industry and earned his masters in data science from IU International University of Applied Sciences.?
Explore more
This article is a beta test. It was created by having an AI generate an initial answer to a question. The response was then fact checked, corrected, and amended by editor Felicia Hou . Any errors or additions? Please let us know in the comments.
Principal Data & Applied Scientist
2 年Algorithms are generally two types, open and closed form. Closed form algorithms can be represented as mathematical equations, for example the equation for Pythagoras theorem or in more complex cases the equation for gradient descent (represented as summation of steps). Open form algorithms are those that Computer Science focuses on, like several of the sorting algorithms. Open form algorithms are broken up to discrete steps which are more logical than mathematical in nature. Note that the final results of both open form and closed form can be achieved via brute force, but the number of states in closed form algorithms will be more as in most cases the state space is the set of real numbers which is infinite. Machine Learning can definitely help us in improve existing algorithms, but mainly for closed form algorithms. This does so by giving us better parameters to feed into the equation, by exploring as infinite state space much more efficiently. However, for open form algorithms an ML approach can definitely give the final output (which can also be achieved by brute force), but it is quite hard to give the discrete steps necessary to 'form' the algorithm.
Data Science & AI Executive
2 年Algorithms are usually inductive or deductive. Either way, they use some form of reductionism (of form, of data) to establish a pragmatic sequence of steps. With machine learning, there are two potential improvements that are possible that may mitigate the extent of reduction involved: 1) Making algorithms more robust, but less accurate: De-noising architectures in cases where data quality is suspect 2) Making algorithms more accurate, but more fragile: Deeper/broader n/w architectures. This may lead to algorithm enhancements too. I have tried this successfully to model risk-weighted safety stock in a multi-echelon supply chain esp. for slow moving items.
Applied AI+Data Researcher at Goldman Sachs | IITK | All Rounder Gold Medallist
2 年Another improvement is ‘Shift in thinking’ for developers - ML techniques give a new way of thinking and collaboration opportunities for developers and statisticians. It’s not just a stats problem anymore to be solved by hard core algorithm thinkers or statisticians, but a data problem. Which data feature is the best indicator - find those, throw in a lot of data and get the linear/non-linear relationships out from an accurate ML model. This way of problem solving is unique. We successfully applied a similar thinking cycle here at GS - https://developer.gs.com/blog/posts/harnessing-machine-learning-improve-data-lake-client-happiness
CIO | Transformative Leader | Building High-Performance Teams | IT Strategy | Innovation | Driving Operational Efficiency | Digital Transformation | Manage Budgets up over $100M and Large Global Teams over 500
2 年Increasing the adaptability of existing algorithms will make them more effective while also increasing their fragile nature. Adaptability needs its parameters and considers when to expand or replace.
Managing Member at NSIP LLC & Member of the Executive Committee of Ellington Healthcare Asset Management LLC
2 年Esteban Valles has the correct outlook which I had intended to say but he did a better job than I would have