Programmable AI Chips and the Next Generation of Intelligent Machines

Thank you for reading my latest article Programmable AI Chips and the Next Generation of Intelligent Machines. Here at LinkedIn and at Forbes I regularly write about management and technology trends.

To read my future articles simply join my network by clicking 'Follow'. Also feel free to connect with me via Twitter, Facebook, Instagram, Podcast or YouTube.

Today, we’re quickly becoming used to using AI-enhanced applications on our phones and PCs. However, although we call these "smart devices," the actual smarts often happen in the cloud, with our devices simply acting as terminals displaying the results.

But as the AI revolution gathers pace, everyday devices will increasingly come with intelligence “baked in” and will perform AI computing locally. This is called edge computing and has a number of benefits, such as reducing the need to transmit large amounts of information over the network, cutting bandwidth costs, and reducing security risks. However, as they run on our phones, vehicles and smart home appliances rather than data centers, edge processors have different requirements. For example, they usually need to be small and power efficient.

This means they will often be optimized for performing a particular task. But designing and manufacturing a new miniature computer for every device or use case is an expensive proposition – enter the FPGA!?

The Field Programmable Gate Array can be thought of as basically a reprogrammable computer chip. They’ve been around since the mid-eighties. Today, however, they’re being infused with AI and are promising to bring intelligence to more devices we use daily in an increasingly flexible and affordable way.

What FPGAi Brings to the Table

Business is changing fast, thanks to shifts in consumer behavior and evolving technology. What is state-of-the-art in AI today will probably look laughably dated within a relatively short timeframe.

This means that chips that can be reprogrammed and repurposed as new AI models are developed, and as new use cases emerge, will make it quicker and easier to deploy new tools and services to market.

Altera – a long standing pioneer of programmable logic and now a subsidiary of Intel – is now trailblazing the field of FPGAi. With a full product line that extends from the highest performing FPGAs for the data center, in the network, and all the way to smart embedded systems at the edge, Altera is meeting new demands with innovative approaches to programmable logic.

I recently spoke with CEO Sandra Rivera about this interesting new paradigm. She told me, “The continued exponential growth in data and the need to process, move, store and secure that data just continues unabated.

“And because of that, more high-performance computing platforms are required, with FPGAs playing a crucial role, because of their processing power, performance and efficiency.”

FPGAi encompasses an ecosystem including Altera’s Agilex 5 FPGA devices – with the only FPGA fabric designed specifically for AI - as well as its “push button” AI design flow, which includes the Quartus, OpenVINO and FPGA AI Suite tools.

“It makes adding AI as simple as adding any other IP to your FPGA solution,” Rivera says.

What this means is that we can expect to see many more devices, from the everyday to the industrial, to potentially life-saving medical use cases, augmented with AI, as building cognitive abilities into machinery becomes quicker and more affordable.

Their ability to handle enormous amounts of data in any form, including video and real-time data, also makes them ideal for use cases involving quality control, manufacturing and autonomous driving.

And as many of these applications will involve collecting and processing sensitive or personal data, it also has huge implications for data security, and the development of “trustworthy” AI.

“You can localize data to train an AI model but then contribute those learnings to a centralized data center so that you’re not compromising someone’s personal information, but you are still contributing to an overall model. This idea of federated AI learning is something that is being utilized more broadly in medical types of applications,” Rivera tells me.

The practical uses for this breakthrough technology are virtually unlimited. From consumer electronics to vast industrial smart factories, it will become simpler and cheaper to bake intelligence in to practically any device.

Who Is Using FPGAi?

While FPGAs are opening new frontiers for intelligent devices, they also have a big role to play in large-scale processing such as AI model training and running huge LLMs (large language models) at a fraction of the cost of GPU-based systems.

One company that’s currently exploring this transformative potential is Positron.ai, who has been using the FPGAi architecture to accelerate large language models (LLMs) based on the transformer neural network architecture ?– ?the technology that underpins hugely powerful and capable apps like ChatGPT. These LLMs enable apps to convert natural human language into computer data that can be processed and converted into new outputs. A fundamental part of the process involves breaking language down into simple bite-sized chunks known as tokens.

CEO Thomas Sohmers explained to me, “Fundamentally, what we saw with Alera’s Agilex 7 line was the ability to access massive amounts of extremely high bandwidth memory and scale up the overall processing capacity by linking together many FPGA accelerators through high-speed SerDes interconnects … this enables us to deal with multi-trillion parameter models and generate tokens very quickly.

“We think this gives us a huge advantage in system building compared to what you’re able to do with the GPUs, CPUs, or any other platform out there, and be able to get to market way faster and with way more flexibility than if we went out and designed a custom ASIC ourselves.”

Another company already leveraging the potential of FPGAs is CriticalLink. It sees the ability of FPGA chips to collect data directly from the source and process it locally as the key to using them to drive competitive advantage for businesses.

“The FPGA is right out there at the edge … collecting image data, collecting video data. And what we’re finding is processing that data in the FPGA right there at the edge … provides a huge advantage for our customers,” VP Tom Catalino tells me.

An example use case for one of its customers might involve detecting defects in a manufacturing environment. Conventionally, the process might involve streaming video of the entire production line to the cloud, where it would be processed by image recognition algorithms to find defective products. However, by applying AI at the edge with FPGA, the data can be processed as it’s collected, significantly reducing the amount of information that needs to be transmitted.

“There’s a cost associated with transmitting that data, there’s cost associated with processing, and then there’s cost associated with pulling the resultant data back out of the cloud. All of this can be reduced,” Catalino says.

FPGAi in Education and the Future of Industrial AI

FPGAi has the potential to accelerate the adoption of industrial AI by providing the power and flexibility that businesses need to innovate more efficiently. And undoubtedly, education will be a key part of the solution when it comes to unlocking this potential.

This is why I was interested to learn about work being carried out at the University of Washington, where applications in telehealth, as well as in the development of personalized learning systems with FPGAi, are being explored.

Post-surgery, FPGAi is being used to capture high-resolution images that can then be processed in 3D, enabling the extraction of precise measurements which clinicians can use to track patient recovery.1

And through a collaboration between the university and Altera, known as LabsLand, learning systems are being developed that can adapt to the individual learning speed and style of different students to improve their ability to absorb and retain information.

Associate teaching professor Dr Rania Hussein told me, “We believe that AI-driven data analytics will help educators find areas where students struggle, allowing prompt and targeted interventions.”

I also asked Dr Hussein about the ongoing role of universities when it comes to educating the innovators of the future.

She told me, "One crucial area is strengthening partnerships with industry leaders. These collaborations will ensure that students have access to the latest technologies and real-world insights, which are essential for professional growth.”

Ongoing collaborative partnerships like this will help businesses and industries to leverage the potential of AI in innovative ways. And with tools such as the FPGAi infrastructure combined with Altera’s Quartus design environment, barriers to participation in this revolution are quickly coming down. In fact, Rivera tells me that 75 percent of Altera’s customers plan to integrate AI into their FPGA applications in the next five years.

The result of this is likely to be that the next generation of truly intelligent machines will more effectively integrate into our lives. This will lead to improved business outcomes, as well as furthering the ongoing transformation of our relationship with technology.

?

1WoundCare management system. Funded by UW Research Royalty Funds and NSF I-Corp.

About Bernard Marr

Bernard Marr is a world-renowned futurist, influencer and thought leader in the fields of business and technology, with a passion for using technology for the good of humanity. He is a best-selling author of over 20 books, writes a regular column for Forbes and advises and coaches many of the world’s best-known organisations.

He has a combined following of 4 million people across his social media channels and newsletters and was ranked by LinkedIn as one of the top 5 business influencers in the world. Bernard’s latest book is ‘Generative AI in Practice’.