New Microchip Technology to Sustain Advances in AI and Machine Learning, and Internet of Things.

An AI accelerator is a class of microprocessor or computer system designed to accelerate artificial neural networks, machine vision and other machine learning algorithms for robotics, internet of things and other data intensive or sensor-driven tasks. They are often many core designs and generally focus on low precision arithmetic.

Computer systems have frequently complemented the CPU with special purpose accelerators for specialized tasks, most notably video cards for graphics, but also sound cards for sound, etc.

As Deep learning and AI workloads rose in prominence, specialize hardware was created or adapted from previous products to accelerate these tasks.

Heterogeneous computing began the incorporation of a number of specialized processors in a single system, or even a single chip, each optimized for a specific type of task.

Architectures such as the Cell microprocessor have features significantly overlapping with AI accelerators including: support for packed low precision arithmetic, data flow architecture, and prioritizing 'throughput' over latency. The Cell microprocessor would go on to be applied to a number of tasks including AI.

CPUs themselves also gained increasingly wide SIMD units (driven by video and gaming workloads) and support for packed low precision data types.

Graphics processing units or GPUs are specialized hardware for the manipulation of images. As the mathematical basis of neural networks and image manipulation are similar, embarrassingly parallel tasks involving matrices.

GPUs became increasingly used for machine learning tasks. As such, as of 2016 GPUs are popular for AI work, and they continue to evolve in a direction to facilitate deep learning, both for training and inference in devices such as self-driving cars. and gaining additional connective capability for the kind of data flow workloads AI benefits from (e.g. Nvidia NVLink). applied to AI acceleration, GPU manufacturers have incorporated neural network specific hardware to further accelerate these tasks. Tensor cores are intended to speed up the training of neural networks.

Deep learning frameworks are still evolving, making it hard to design custom hardware. Reconfigurable devices like field-programmable gate arrays (FPGA) make it easier to evolve hardware, frameworks and software alongside each other.

MIT scientists, developed a new highly efficient chip that may enable mobile devices to run powerful artificial intelligence algorithms, and help usher in the "Internet of things".

Internet of Things (IoT) connects billions of devices in an Internet-like structure. Each device encapsulated as a real-world service in which provides functionality and exchanges information with other devices. This large-scale information exchange results in novel interactions between things and people.

Artificial intelligence is coming to your Computer, tablet and mobile phone, a Neural Engine as part of its A11 Bionic chip; the Huawei Kiri 970 chip has what's called a Neural Processing Unit or NPU on it; and the Pixel 2 has a secret AI-powered imaging chip.

So what exactly are these next-gen chips designed to do?

As mobile chip sets have grown smaller and more sophisticated, they have started to take on more jobs and more different kinds of jobs.

Case in point, integrated graphics -- GPUs now sit alongside CPUs at the heart of high-end smartphones, handling all the heavy lifting for the visuals so the main processor can take a breather or get busy with something else.

The new breed of AI chips are very similar - only this time the designated tasks are recognizing pictures of your pets rather than rendering photo-realistic FPS backgrounds.

What is Artificial Intelligence

Artificial intelligence (AI) is a segment of computer science that typically refers to the creation of machines or computers capable of intelligent behavior approximating that of human beings. The term was coined by Stanford researcher John McCarthy in 1956 at The Dartmouth Conference.

Modern understanding of AI can also denote intelligence exhibited by machines or computers, i.e., intelligent machines. These devices can perform cognitive functions like learning, problem solving, or using language that are usually associated with humans.

A person hidden behind a screen operating levers on a mechanical robot is artificial intelligence in the broadest sense of course today's AI is way beyond that, but having a programmer code responses into a computer system is just a more advanced version of getting the same end result (a robot that acts like a human).

As for computer science and the smartphones in your pocket, here AI tends to be more narrowly defined. In particular it usually involves machine learning, the ability for a system to learn outside of its original programming, and deep learning, which is a type of machine learning that tries to mimic the human brain with many layers of computation.

Those layers are called neural networks, based on the neural networks inside our heads.

So machine learning might be able to spot a spam message in your inbox based on spam it's seen before, even if the characteristics of the incoming email weren't originally coded into the filter it's learned what spam email is.

Deep learning is very similar, just more advanced and nuanced, and better at certain tasks, especially in computer vision the "deep" bit means a whole lot more data, more layers, and smarter weighting. The most well-known example is being able to recognize what a dog looks like from a million pictures of dogs.

Plain old machine learning could do the same image recognition task, but it would take longer, need more manual coding, and not be as accurate, especially as the variety of images increased. With the help of today's super powered hardware, deep learning

(a particular approach to machine learning, remember), is much better at the job.

Apple introduces its Neural Engine.

To put it another way, a machine learning system would have to be told that cats had whiskers to be able to recognize cats. A deep learning system would work out that cats had whiskers on its own.

AI chips on smartphones

AI chips are doing exactly what GPU chips do, only for artificial intelligence rather than graphics offering a separate space where calculations particularly important for machine learning and deep learning can be carried out.

As with GPUs and 3D graphics, AI chips give the CPU time to focus on other tasks, and reduces battery draw at the same time. In also means your data is more secure, because less of it has to be sent off to the cloud for processing.

So what does this mean in the real world

It means image recognition and processing could be a lot faster. For instance, Huawei claims that its NPU can perform image recognition on 2,000 pictures every second, which the company also claims is 20 times faster than it would take with a standard CPU.

The Huawei Kirin 970 has a dedicated AI component. More specifically, it can perform 1.92 teraflops, or a trillion floating point operations per second, when working with 16-bit floating point numbers.

As opposed to integers or whole numbers, floating point numbers - with decimal points - are crucial to the calculations running through the neural networks involved with deep learning.

Apple calls its AI chip, part of the A11 Bionic chip, the Neural Engine. Again, it's dedicated to machine learning and deep learning tasks recognizing your face, recognizing your voice, recording animojis, and recognizing what you're trying to frame in the camera.

It can handle some 600 billion operations per second, Apple claims. App developers can tap into this through Core ML, and easy plug-and-play way of incorporating image recognition and other AI algorithms.

Core ML doesn't require the iPhone X to run, but the Neural Engine handles these types of tasks faster. As with the Huawei chip, the time spend offloading all this data processing to the cloud should be vastly reduced, theoretically improving performance and again lessening the strain on battery life.

AI chips, recognising faces now, and much more soon. And that's really what these chips are about: Handling the specific types of programming tasks that machine learning, deep learning, and neural networks rely on, on the phone, faster than the CPU or GPU can manage. When Face ID works in a snap, you've likely got the Neural Engine.

Is this the future

Will all smartphone inevitably come with dedicated AI chips in future? As the role of artificial intelligence on our handsets grows, the answer is likely yes.

Qualcomm chips can already use specific parts of the CPU for specific AI tasks, and separate AI chips is the next step. Right now these chips are only being utilized for a small subsection of tasks, but their importance is going to only grow.

Specialized Chips

For over 50 years, a foundational principle behind the development of microprocessors in computer chips has been Moore’s Law. This law is an observation made by Intel co-founder Gordon Moore back in the 1960s, which assumes that the number of transistors in an integrated circuit doubles roughly every 18 months initially, 24 months effectively increasing microchip complexity.

The problem is, Moore’s Law is nearing its end, as transistors can no longer be effectively miniaturized to increase chip performance.

This predicament, coupled with the need for higher levels of processing power, presents a hurdle that must be overcome in the continued development of artificial intelligence (AI).

The Defense Advanced Research Projects Agency (DARPA), the research arm of the U.S. Defense Department, thinks that developing specialized circuits — or application-specific integrated circuit chips (ASICs) — is one of the ways to overcome these limitations.

One project is the Software Defined Hardware, which is developing “a hardware/software system that allows data-intensive algorithms to run at near ASIC efficiency without the cost, development time or single application limitations associated with ASIC development.” The second project is called Domain-Specific System on a Chip.

Simply put, this takes a combined approach of using general purpose chips, hardware coprocessors, and ASICs, “into easily programmed [systems on a chip] for applications within specific technology domains.”

AI's Limits

Experts generally agree that Moore’s Law will no longer be viable by the 2020s. Meanwhile, what AI is being trained to do requires a great deal more processing power — close to what the human brain is capable of. Increasing processing power, something quantum computing and IBM’s neurosurgical chips promise, will be crucial to continuing the development of AI.

For DARPA, it’s a problem that needs a more immediate solution. Artificially intelligent systems are being trained to think like human beings do, or even better. Yet, the human brain’s ability to process information and data remains unparalleled, even by today’s best artificial neural networks which depend on the computing power of microchips.

The smaller, the better

Transistors are semiconductors that work as the building blocks of modern computer hardware. Already very small, smaller transistors are an important part of improving computer technology.

Current transistors in use are in 14nm scale technology, with 10nm semiconductors expected in 2017 or 2018, supposedly in Intel’s Cannonlake line a trend following Intel co-founder Gordon Moore’s prediction that transistor density on integrated circuits would double every two years, improving computer electronics. Berkeley Lab’s team seems to have beaten them into it, developing a functional 1nm transistor gate.

Keeping up with Moore's Law

“We made the smallest transistor reported to date. The gate length is considered a defining dimension of the transistor. We demonstrated a 1-nanometer-gate transistor, showing that with the choice of proper materials, there is a lot more room to shrink our electronics.

Silicon-based transistors function optimally at 7nm but fail below 5nm, where electrons start experiencing a severe short channel effect called quantum tunneling.

Supposedly, silicon allows for lighter electrons, moving with less resistance. This, however, makes 5nm gates too thin to control electron flow and keep them in the intended logic state. “The Berkeley Labs team found a better material in molybdenum disulfide (MoS2). Electrons flowing through MoS2 are heavier, making them easier to control even at smaller gate sizes. MoS2 is also more capable of storing energy in an electric field.

Combined with carbon nanotubes with diameters as small as 1nm, this allowed for the shortest transistors ever.

source: Lawrence Cummins, References: Defense One, DARPA, & News Center, Science.