A radical new technique lets AI learn with practically no data

Machine learning typically requires a lot of training data and many examples. If you want an AI model to recognise a cat, you need to show it hundreds, if not thousands, of images of different cats. 

This current process is expensive and time consuming.

The human brain works very differently and usually only needs a few examples before being able to recognise it for life. 

Sometimes children don’t need any examples to identify something! When shown photos of a horse and a rhino, then told that a unicorn is something in between, children can recognise the mythical creature in a picture book the first time they see it. 

This remarkable learning pattern is the holy grail for machine learning.

Source: MS TECH / PIXABAY

A new paper from the University of Waterloo in Ontario suggests that AI models should also be able to do this. The AI model should be able to accurately recognise more objects than the number of examples it was trained on. 

They call this process “less than one-shot” learning, and it could be a game changer for artificial learning. 

How “less than one”-shot learning works

The essence of how “less than one”-shot learning works is to take giant data sets and compress them into much smaller ones. 

In the research study, the team took the MNIST database, which contains 60,000 training images of handwritten digits from 0 to 9. They then carefully selected a much smaller number of images, just 10 images.

Sample images from the MNIST dataset

Sample images from the MNIST dataset. Source: WIKIMEDIA

The 10 images selected were carefully chosen to try to contain an equivalent amount of information to the full set. When the model was trained on just these 10 images, instead of the full 60,000, it achieved nearly the same accuracy as a model trained on the full data set!

How could this happen?

The researchers realised that the trick was to create blends of numbers together, then create “soft labels,” drawing on the idea of the horse and rhino having the features of a unicorn.

“If you think about the digit 3, it kind of also looks like the digit 8 but nothing like the digit 7,” says Ilia Sucholutsky, a PhD student at Waterloo and lead author of the paper. 

“Soft labels try to capture these shared features. So instead of telling the machine, ‘This image is the digit 3,’ we say, ‘This image is 60% the digit 3, 30% the digit 8, and 10% the digit 0.’”

Is this the holy grail of machine learning?

What the researchers demonstrate in their latest paper is a purely mathematical exploration that only applies to classification problem areas. However, it’s a promising step forward. With more complicated areas of learning, much more work needs to be done.

Trying to use the human brain as a learning pattern is an interesting approach because of the huge capacity we have for learning quickly. 

While this approach is not yet ready for use in most corporate scenarios, it could trigger the next wave of AI machine learning development. The jury is still out on this.

This article was inspired by: